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Overview 


Introduction 


Children develop fastest in their earliest years, and the skills and abilities they develop 
in those years help lay the foundation for future success. Early negative experiences 
can contribute to poor social, emotional, cognitive, behavioral, and health outcomes both 
in early childhood and in later life. One approach that has helped parents and their young 
children is home visiting, which provides individually tailored support, resources, and 
information to expectant parents and families with young children. Many early childhood 
home visiting programs work with low-income families to help ensure the healthy devel- 
opment and well-being of their children. 


In 2010, Congress authorized the Maternal, Infant, and Early Childhood Home Visiting 
(MIECHV) Program by enacting section 511 of the Social Security Act, 42 U.S.C. § 711, 
which also appropriated funding for fiscal years 2010 through 2014. Subsequently en- 
acted laws extended funding for the program through fiscal year 2022. The program is 
administered by the Health Resources and Services Administration (HRSA) in collabo- 
ration with the Administration for Children and Families (ACF) within the U.S. Depart- 
ment of Health and Human Services (HHS). The initiation of the MIECHV Program be- 
gan a major expansion of evidence-based home visiting programs for families living in 
at-risk communities. The legislation authorizing MIECHV recognized that there was con- 
siderable evidence about the effectiveness of home visiting, but also required an evalu- 
ation of MIECHV in its early years, which became the Mother and Infant Home Visiting 
Program Evaluation (MIHOPE). The overarching goal of MIHOPE is to learn whether 
families and children benefit from MIECHV-funded early childhood home visiting pro- 
grams, and if so, how. MIHOPE includes the four evidence-based home visiting models 
that 10 or more states chose in their fiscal year 2010-2011 plans for MIECHV funding: 
Early Head Start - Home-based option, Healthy Families America, Nurse-Family Part- 
nership, and Parents as Teachers. From October 2012 to October 2015, a total of 4,229 
families entered the study. 


Given the positive effects found in previous long-term studies of home visiting and pre- 
vious findings that the benefits of home visiting outweigh the costs only after children 
enter elementary school, ACF and HRSA initiated plans to design long-term follow-ups 
with the families who are participating in MIHOPE. Under contract with ACF, MDRC is 
conducting this work in partnership with Columbia University and Mathematica Policy 
Research. ACF and HRSA were interested in ensuring that any additional follow-ups 
build on information from the earlier waves of data collection to the greatest extent 


possible, and that any proposed follow-up points build on one another. This long-term 
follow-up phase is called MIHOPE-LT. This report presents the proposed design for po- 
tential long-term follow-ups with MIHOPE families through the time when their children 
are in high school. The report also presents the detailed design for the follow-up that is 
occurring when children are in kindergarten. 


Primary Research Questions 


The four primary research questions that the long-term follow-ups were designed to ad- 
dress are: 


1. What are the long-term effects of being assigned to receive evidence-based 
home visiting for families who enrolled in MIHOPE? 


2. Are the long-term effects of home visiting larger for some families than for 
others? 


3. What are the pathways through which home visiting affects families’ longer- 
term outcomes? 


4. How do the monetary benefits of home visiting compare with its costs over 
the long term? 


Purpose 


Several previous studies of the four home visiting models included in MIHOPE have 
provided information on the long-term effects of home visiting programs. MIHOPE-LT 
can expand this body of evidence. The previous studies had relatively small samples, 
were model-specific, and did not examine the same outcomes in the same way across 
models, making it difficult to summarize across studies and models. In contrast, 
MIHOPE-LT will measure the same outcomes for all four evidence-based models in- 
cluded in MIHOPE. In addition, most of the previous long-term studies were completed 
many years ago. Home visiting programs have changed over time, both because of 
statutory requirements for federal funding through the MIECHV Program and because 
programs and models are continually evolving through quality-improvement efforts. 
Moreover, the context in which the programs operate, and the program participants, 
have also changed. As programs evolve and program context changes, additional eval- 
uation can determine whether programs continue to be effective in meeting their goals. 


Key Findings and Highlights 


MIHOPE estimated the effects of MIECHV-funded early childhood home visiting pro- 
grams on family and child outcomes around the time children were 15 months of age 
and found small positive effects for families across several outcome areas. (See Charles 
Michalopoulos, Kristen Faucetta, Carolyn J. Hill, Ximena A. Portilla, Lori Burrell, Helen 
Lee, Anne Duggan, and Virginia Knox, /mpacts on Family Outcomes of Evidence-Based 
Early Childhood Home Visiting: Results from the Mother and Infant Home Visiting Pro- 
gram Evaluation, OPRE Report 2019-07.) Contact with the MIHOPE families was main- 
tained via short surveys that were completed around the time children were 2.5 and 3.5 
years of age. 


The MIHOPE-LT study team identified the four primary research questions listed above 
and proposed a study design that could be used to answer these questions. A follow-up 
when children are in kindergarten began in January 2019, and the plans for this data 
collection are discussed in detail in this report. Three other potential follow-up time points 
based on the participating child’s expected progression through school were also iden- 
tified: third grade, middle school, and high school. Obtaining information about families’ 
well-being over time might be particularly important for answering the question of how 
the monetary benefits of home visiting compare with its costs, because benefits may 
continue to accrue as children get older. This report does not present detailed plans for 
follow-ups past kindergarten. Detailed study designs would need to be developed in the 
future if follow-up at later time points was to be conducted. 


Methods 


MIHOPE included 88 local home visiting programs in 12 states. More than 4,200 
women who were pregnant or had children younger than six months of age were ran- 
domly assigned to a MIECHV-funded home visiting program or to a control group who 
received information about other appropriate services in the community. 


For the kindergarten follow-up, data collection methods are similar to those used for the 
MIHOPE follow-up that occurred when children were 15 months of age. Specifically, 
information is being gathered from a structured interview conducted with mothers. The 
study is also drawing on video-recorded interactions of mothers and children playing 
with toys; interviewer observations of parental warmth and children’s self-regulation; di- 
rect assessments of children’s language skills, math skills, and executive function; direct 
assessments of mothers’ executive function; state administrative child welfare data; 
state school records data; federal administrative Medicaid data; federal employment and 


earnings data from the National Directory of New Hires; and a survey conducted with 
children’s teachers. 
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Executive Summary 


Children develop fastest in their earliest years, and the skills and abilities they develop 
in those years lay the foundation for their future success. Similarly, early negative ex- 
periences can contribute to poor social, emotional, cognitive, behavioral, and health out- 
comes both in early childhood and in later life. Children growing up in poverty tend to be 
at greater risk of encountering adverse experiences that negatively affect their develop- 
ment. One approach that has helped is home visiting, which provides individually tailored 
support, resources, and information to expectant parents and families with young chil- 
dren. Many early childhood home visiting programs aim to support the healthy develop- 
ment of infants and toddlers and work with low-income families in particular to help en- 
sure their well-being. 


In 2010, Congress authorized the Maternal, Infant, and Early Childhood Home 
Visiting (MIECHV) Program by enacting section 511 of the Social Security Act, 42 U.S.C. 
§ 711, which also appropriated funding for fiscal years 2010 through 2014.2 Subse- 
quently enacted laws extended funding for the program through fiscal year 2022.2 The 
program is administered by the Health Resources and Services Administration (HRSA) 
in collaboration with the Administration for Children and Families (ACF) within the U.S. 
Department of Health and Human Services. The initiation of the MIECHV Program be- 
gan a major expansion of evidence-based home visiting programs for families living in 
at-risk communities. 


The legislation authorizing MIECHV recognized that there was considerable ev- 
idence about the effectiveness of home visiting, but also required an evaluation of 
MIECHV in its early years,4 which became the Mother and Infant Home Visiting Program 
Evaluation (MIHOPE). The overarching goal of MIHOPE is to learn whether families and 
children benefit from MIECHV-funded early childhood home visiting programs, and if so, 
how.® 


‘National Research Council and Institute of Medicine (2000). 

Social Security Act of 1935. SEC. 511 [42 U.S.C. 711] (j) (1) (2010). 

3Funds for subsequent fiscal years were appropriated by section 209 of the Protecting Access to 
Medicare Act of 2014, Pub. L. 113-93 (fiscal year 2015); section 218 of the Medicare Access and 
Children’s Health Insurance Program Reauthorization Act of 2015, Pub. L. 114-10 (fiscal years 2016- 
2017); and section 50601 of the Bipartisan Budget Act of 2018, Pub. L. 115-123 (fiscal years 2018- 
2022). 

4Social Security Act of 1935. SEC. 511 [42 U.S.C. 711] (g) (2) (2010). 

SMIHOPE is studying those programs as they operated from 2012 through 2017. 
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Given the positive effects found in previous long-term studies of home visiting, 
as well as previous findings that the benefits of home visiting outweigh the costs only 
after children enter elementary school,* ACF and HRSA initiated plans to design long- 
term follow-ups with the families who are participating in MIHOPE. Under contract with 
ACF, MDRC is conducting this work in partnership with Columbia University and Math- 
ematica Policy Research. This long-term follow-up phase is called MIHOPE-LT, and the 
study design is the subject of this report. 


The purpose of this design phase was to determine the most fruitful times to 
collect data to answer questions of interest in the context of a study that follows families 
over time. A study that follows families over time provides an opportunity to examine 
child and family outcomes at individual time points as children get older, and to learn 
about the trajectories of child and family outcomes. ACF and HRSA were interested in 
ensuring that any additional follow-ups build on information from the earlier waves of 
data collection to the greatest extent possible, and that any proposed follow-up points 
build on one another.’ 


MIHOPE-LT: Context and Goals 


Several previous studies of the four home visiting models included in MIHOPE have 
provided information on the long-term effects of home visiting programs.? MIHOPE-LT 
can expand this body of evidence. The previous studies had relatively small samples 
(most included fewer than 1,000 families), were model-specific, and did not examine the 
same outcomes in the same way across models, making it difficult to summarize across 
studies and models. In contrast, MIHOPE-LT will measure the same outcomes for the 
four evidence-based models included in MIHOPE: Early Head Start - Home-based op- 
tion, Healthy Families America, Nurse-Family Partnership, and Parents as Teachers. In 
addition, most of the previous long-term studies were completed many years ago.° 
Home visiting programs have changed over time, both because of statutory require- 
ments for federal funding through the MIECHV Program, and because programs and 
models are continually evolving through quality improvement efforts. Moreover, the con- 
text in which the programs operate, and the program participants, have also changed. 


®See Michalopoulos, Faucetta, Warren, and Mitchell (2017). 

7MIHOPE had already collected data from families when children were 15 months of age, 2.5 
years of age, and 3.5 years of age. 

8See Michalopoulos, Faucetta, Warren, and Mitchell (2017). 


°Most of the studies began enrolling families before 1995, and most follow-ups occurred before 
2005. 
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As programs evolve and program context changes, additional evaluation can determine 
whether programs continue to be effective in meeting their goals. 


The primary goal of MIHOPE-LT is to measure the long-term effects of home 
visiting programs on family outcomes. To that end, the study team aimed to propose a 
study design that will try to answer these primary research questions: 


1. What are the long-term effects of being assigned to receive evidence- 
based home visiting for families who enrolled in MIHOPE? 


2. Are the long-term effects of home visiting larger for some families than 
for others? 


3. What are the pathways through which home visiting affects families’ 
longer-term outcomes? 


4. How do the monetary benefits of home visiting compare with its costs 
over the long term? 


The next section describes the original MIHOPE design in order to familiarize 
readers with the foundation for MIHOPE-LT. 


Background: The MIHOPE Design 


MIHOPE is a randomized controlled trial. That is, to provide reliable estimates of home 
visiting programs’ effects, women who enrolled in the study were randomly assigned to 
a MIECHV-funded local home visiting program, or to a control group who received infor- 
mation about other appropriate services in the community. 


MIHOPE included 88 local home visiting programs in 12 states: California, Geor- 
gia, Illinois, lowa, Kansas, Michigan, Nevada, New Jersey, Pennsylvania, South Caro- 
lina, Washington, and Wisconsin. States were selected based on a number of criteria, 
including whether they planned to implement more than one of the four evidence-based 
models that MIHOPE included and to support five eligible local programs or more, 
whether they contributed geographic diversity to the sample, and whether they contrib- 
uted some local programs operating in nonmetropolitan areas to the final sample. 


The 88 local programs that participated in MIHOPE consisted of 19 Early Head 
Start programs, 26 Healthy Families America programs, 22 Nurse-Family Partnership 
programs, and 21 Parents as Teachers programs. As was true for states, local programs 
also had to meet several criteria to be included in MIHOPE, such as having been in 
operation for at least two years when they entered the study and being able to recruit 
enough families to fill the program slots and allow for a randomly chosen control group. 
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Characteristics of MIHOPE Families 


A total of 4,229 families entered the study from October 2012 to October 2015. 
In order to be eligible for MIHOPE, women had to be at least 15 years of age, be either 
pregnant or have a child younger than 6 months of age when they enrolled in the study, 
be able to speak English or Spanish well enough to provide consent and complete a 
survey when they entered the study, and not be receiving home visiting services from a 
participating local program already. They also had to be interested in receiving home 
visiting services and had to meet the local program’s eligibility criteria. 


Women participating in MIHOPE tended to be young, economically disadvan- 
taged, and racially and ethnically diverse, and they exhibited a variety of risks at study 
entry that could affect their children’s development. Almost two-thirds of the women were 
younger than 25 years of age, and 35 percent were younger than 21 years of age. Forty- 
two percent of the women in the sample did not have high school diplomas; as might be 
expected, older women in the sample were more likely to have completed high school. 
Nearly 75 percent of women in the sample were receiving benefits from the Special 
Supplemental Nutrition Program for Women, Infants, and Children, and more than half 
were enrolled in the Supplemental Nutrition Assistance Program. More than half of the 
women reported that their households had experienced food insecurity in the past year 
(meaning there were times when they worried about food or ran out of it), nearly one- 
third reported substance use before pregnancy, over two-fifths reported symptoms of 
either depression or anxiety, and about one-fifth reported experiencing or perpetrating 
physical acts of intimate partner violence. 


Early Effects on MIHOPE Families 


The first follow-up phase of MIHOPE included an impact analysis to estimate the effects 
of MIECHV-funded home visiting programs in a broad range of outcome areas men- 
tioned in the authorizing legislation and for different subgroups of families, using data 
that were gathered when children were about 15 months of age.'® Effects were esti- 
mated in the following outcome areas: (1) prenatal, maternal, and newborn health; (2) 
child health and development, including child maltreatment; (3) parenting skills; (4) crime 
or domestic violence; (5) family economic self-sufficiency; and (6) referrals and service 
coordination." 


10Michalopoulos et al. (2019) describes the results of the impact analysis and analysis of impact 
variation from the first phase of MIHOPE. 

MSEC. 511 [42 U.S.C. 711] (d) (2) (B). The legislation also indicated that programs should im- 
prove school readiness and academic achievement, but children in MIHOPE were too young to pro- 
vide information about that area at the follow-up that occurred when they were 15 months of age. 
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The impact analysis when children were 15 months of age found that there were 
positive effects of home visiting for families in MIHOPE, and that most estimated effects 
were similar to but somewhat smaller than those found in past studies of individual home 
visiting models. However, it is important to note that MIHOPE differs from those studies 
in many respects. For example, most of those studies were conducted in a single local 
area rather than including locations across the country, and some were conducted many 
years ago, when similar services were less likely to be available to control group families. 
Estimated effects are statistically significant for 4 of the 12 confirmatory outcomes: the 
quality of the home environment, the frequency of psychological aggression toward the 
child, the number of Medicaid-paid child emergency department visits, and child behav- 
ior problems. ‘'? Overall, for 9 of the 12 confirmatory outcomes, program group families 
fared better than control group families on average, which is unlikely to have occurred 
for the study sample if the home visiting programs made no true difference in family 
outcomes. In addition, results for clusters of exploratory outcomes suggest that home 
visiting may improve maternal health and might reduce household aggression. *? 


Checking in with Families When Children Are Preschool Age 


In addition to following up with the MIHOPE families when the study child was 15 months 
of age, follow-up occurred at two later points in time: (1) when the child was 2.5 years of 
age, and (2) when the child was 3.5 years of age. The phase of MIHOPE that includes 
these two later points is called MIHOPE Check-in. Data collection for this phase began 
in September 2015 and concluded in June 2019. 


MIHOPE Check-in included brief surveys to gather information from parents 
about child and family well-being. Information about these outcomes allows the study 
team to estimate ongoing effects of home visiting as children grow older. Updated con- 
tact information was also obtained at each point during the MIHOPE Check-in phase in 
preparation for the MIHOPE long-term follow-up. Because the MIHOPE Check-in data 


12To focus the analysis on areas where home visiting programs were likely to have their greatest 
short-term effects, the study team chose 12 outcomes based on the evidence of effects from the four 
evidence-based models included in MIHOPE that existed before the analysis began, the policy rele- 
vance of those outcomes, and the quality of the tools available to measure the outcomes. Following 
the terminology used in a report written for the Institute of Education Sciences, the 12 outcomes are 
considered “confirmatory.” See Schochet (2008). 

13Exploratory outcomes capture other aspects of the areas the legislation intended home visiting 
to improve. These outcomes were considered exploratory because past home visiting studies had not 
found effects on them or they had not been examined in previous studies. They were still thought to 
be areas where MIECHV-funded programs might improve family outcomes. 


Household aggression includes experiences of intimate partner violence and child maltreatment. 
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were still being collected when the long-term follow-up study was designed, they did not 
contribute to the MIHOPE-LT design. 


MIHOPE-LT Kindergarten Follow-Up 


The next follow-up with MIHOPE families is occurring when children are in kindergarten. 
Children of MIHOPE families are attending kindergarten in four cohorts, from the 2018- 
2019 school year to the 2021-2022 school year. 


Measuring children’s cognitive, behavioral, self-regulatory, and social-emotional 
skills before formal schooling begins or at the outset of formal schooling will provide 
important data on intermediate effects of home visiting. In addition, a wealth of literature 
demonstrates that children’s math, language, and social-emotional skills at the time of 
the transition to formal schooling are predictive of academic and behavioral outcomes 
over the longer term, '* and a follow-up during the kindergarten year will allow the study 
team to measure these key mediators. Consistent with this research evidence, the leg- 
islation that authorized MIECHV indicated that home visiting programs are expected to 
improve school readiness. *® 


The study team identified eight areas of adult and child functioning and behavior 
where effects of home visiting services are most likely to be observed when children are 
kindergarten age: 


e Family economic self-sufficiency 

e Maternal positive adjustment*é 

e Maternal behavioral health’’ 

e Family environment and relationship between parents 
e Parent-child relationship and interactions 


e Parental support for child’s cognitive development 


"4Duncan et al. (2007); Eisenberg, Valiente, and Eggum (2010); Portilla et al. (2014). 

18SEC. 511 [42 U.S.C. 711] (c) (1) indicates that grants are to be made to enable eligible entities 
to deliver home visiting services in order to promote improvement in several outcome areas that in- 
clude school readiness. SEC. 511 [42 U.S.C. 711] (d) (1) (A) includes school readiness in the list of 
benchmark areas that eligible entities are expected to improve. 


16The term “maternal positive adjustment’ is used here to refer to aspects of maternal functioning 
such as parenting stress, mastery, self-regulation, and household chaos. 


‘7This area includes maternal mental health and maternal substance use and alcohol use. 
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e Child social, emotional, and cognitive functioning and school readi- 
ness 


e Receipt of and connection to services 


Two additional areas (social support; school and neighborhood context) are be- 
ing measured at the kindergarten follow-up primarily because they can provide important 
context and information about characteristics that may moderate the long-term impacts 
of home visiting. Table ES.1 shows all the areas that will be examined at the kinder- 
garten follow-up. 


At the kindergarten follow-up, the study team plans to conduct an impact analysis 
and mediational analyses. The impact analysis will assess the effectiveness of MIECHV- 
funded early childhood home visiting programs in improving the outcomes of families 
and children when the study child is in kindergarten, both overall and across key sub- 
groups of families and programs. Mediational analyses will be conducted to shed light 
on the pathways through which home visiting has longer-term effects on families. In 
other words, the study will look at the relationships between earlier outcomes (from the 
15-month, 2.5-year, and 3.5-year follow-ups) and outcomes when the child is in kinder- 
garten. 


MIHOPE-LT Follow-Up Points After Kindergarten 


As indicated above, the study team was contracted to design long-term follow-ups that 
could build on information from earlier waves of data collection and build on one another. 
Three other potential follow-up points were also identified through the literature reviews 
and consultations with experts conducted by the MIHOPE-LT study team: third grade, 
middle school, and high school. This report briefly describes the rationale for data col- 
lection at these three later time points, but does not present detailed plans for follow-ups 
past kindergarten. 


In addition, brief check-ins with families (to obtain updated contact information 
and maintain engagement with the study) could occur periodically, and administrative 
data could be obtained throughout the follow-ups and could be collected past the last 
follow-up with families. 


Obtaining information about families’ well-being over time might be particularly 
important for answering the question of how the monetary benefits of home visiting 
compare with its costs. Because benefits may continue to accrue as children get older, 
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Table ES.1 


MIHOPE-LT Kindergarten Constructs 


Outcome Area Construct 
Family economic self- e Public assistance receipt e Food insecurity 
sufficiency e Employment and earnings e Housing status and mobility 
e Income e Highest level of education 
e Material hardship e Subsequent pregnancies and births 


Maternal positive 
adjustment 


Maternal behavioral health 


Family environment and 
relationship between 
parents 


Parent-child relationship 
and interactions 


Parental support for child’s 
cognitive development 


Child functioning (school 
readiness) 


Receipt of and connection 


to services 


Social support 


School and neighborhood 
context 


Mastery 

Mobilizing resources 
Parenting stress 
Parent-child separations 


Depressive symptoms 
Drug use 


Mother's relationship status 
Mother’s relationship with biological 
father of child 


e Parental warmth 
e Parent-child interaction 
e Abuse (physical, sexual) 


Home literacy environment 


e Behavior problems 
e Social-emotional skills 


e Learning behaviors and approaches to 


e Child care setting before kindergarten 


learning 


Child received any early intervention 
services 


e Child has health insurance coverage 


Involvement of the biological father 
or father figure with the child 


School characteristics 


e Household chaos 
e Self-regulation (working memory) 
e Child school attendance and 


tardiness 


Alcohol use 


Family conflict 

Physical violence: perpetration 
Physical violence: victimization 
Experience with battering 


e Psychological aggression 


Neglect 


Cognitive stimulation 


Disciplinary incidents 
Executive function 
Math skills 
Language skills 


Child emergency department visits 


e Child hospitalizations 


Child receiving any special 
education services/has an 
individualized education program 


Social support 


Neighborhood disadvantage 


obtaining information about family well-being after the children’s kindergarten year might 
allow the study to measure more of the benefits. It is likely that a benefit-cost analysis 
could be conducted at the kindergarten follow-up point, but using the kindergarten data 
might require projecting the value of benefits that accrue later than kindergarten or 
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measuring only the benefits that accrue through kindergarten. Both are accepted 
strategies, but either could limit how precisely the study could answer the question of 
how the monetary benefits of home visiting compare with its costs. 


Contributions of MIHOPE-LT 


MIHOPE-LT will allow for the examination of long-term effects of MIECHV-funded home 
visiting programs and can expand the evidence from previous long-term studies of home 
visiting programs. MIHOPE-LT will also build on the evidence from the first follow-up 
with MIHOPE families, which occurred when children were about 15 months of age and 
provided information on the short-term effects of MIECHV-funded home visiting pro- 
grams. It will also build on the data from the brief MIHOPE Check-in surveys that were 
collected when children were about 2.5 years of age and 3.5 years of age. Data from 
these surveys were not yet analyzed at the time MIHOPE-LT was designed. Additional 
follow-ups can allow the same constructs to be measured at multiple time points so that 
effects in particular areas can be more fully understood. Additional follow-up also ena- 
bles the examination of constructs that were not, and in some cases could not be, meas- 
ured when children were 15 months, 2.5 years, or 3.5 years of age. 


MIHOPE-LT can provide information about how small, measurable changes in 
particular areas of adult and child functioning that were the result of an early childhood 
intervention affect longer-term well-being. Information about these connections could be 
used by policymakers and researchers who are interested in the life trajectories of chil- 
dren and families. 


Home visiting programs intervene early in the lives of children whose families 
face a variety of risk factors because these programs aim to improve the long-term well- 
being of at-risk children. MIHOPE-LT will build evidence about these intended long-term 
effects and will provide information on whether and how home visiting might have 
changed the life course of MIHOPE families. 
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Chapter 1 


Introduction 


Children develop fastest in their earliest years, and the skills and abilities they develop 
in those years lay the foundation for future success.‘ Similarly, early negative experi- 
ences can contribute to poor social, emotional, cognitive, behavioral, and health out- 
comes both in early childhood and in later life. Children growing up in poverty tend to be 
at greater risk of encountering adverse experiences that negatively affect their develop- 
ment. One approach that has helped is home visiting, which provides individually tailored 
support, resources, and information to expectant parents and families with young chil- 
dren. Many early childhood home visiting programs aim to support the healthy develop- 
ment of infants and toddlers and work with low-income families in particular to help en- 
sure their well-being. 


Home visiting programs in the United States have their origins in the late nine- 
teenth century, when charitable organizations used home visiting to try to reduce poverty 
by changing the behavior of the urban poor.2 Home visiting later expanded to include 
approaches such as visits by public health nurses to promote infant and child health, 
Head Start home visiting to promote child development, and home-based family support 
to promote positive parenting and prevent child maltreatment.? As currently practiced, 
home visitors identify family strengths, needs, concerns, and interests and attempt to 
address those in partnership with families through education and support during home 
visits or through referrals to and coordination with community services. 


In 2010, Congress authorized the Maternal, Infant, and Early Childhood Home 
Visiting (MIECHV) Program by enacting section 511 of the Social Security Act, 42 U.S.C. 
§ 711, which also appropriated funding for fiscal years 2010 through 2014.4 Subse- 
quently enacted laws extended funding for the program through fiscal year 2022.5 The 
program is administered by the Health Resources and Services Administration (HRSA) 
in collaboration with the Administration for Children and Families (ACF) within the U.S. 


‘National Research Council and Institute of Medicine (2000). 

*Weiss (1993). 

3Combs-Orme, Reis, and Ward (1985); Harding et al. (2007); Love et al. (2005). 

4SEC. 511 [42 U.S.C. 711] (j) (1). 

5Funds for subsequent fiscal years were appropriated by section 209 of the Protecting Access to 
Medicare Act of 2014, Pub. L. 113-93 (fiscal year 2015); section 218 of the Medicare Access and 
Children’s Health Insurance Program Reauthorization Act of 2015, Pub. L. 114-10 (fiscal years 2016- 
2017); and section 50601 of the Bipartisan Budget Act of 2018, Pub. L. 115-123 (fiscal years 2018- 
2022). 


Department of Health and Human Services (HHS).° The initiation of the MIECHV Pro- 
gram began a major expansion of evidence-based home visiting programs for families 
living in at-risk communities. 


The legislation authorizing MIECHV recognized that there was considerable ev- 
idence about the effectiveness of home visiting, but also called for research to increase 
knowledge about the implementation and effectiveness of home visiting.” States that 
receive MIECHV funding are required to devote the majority of their MIECHV funding to 
the delivery of services according to the specifications of designated evidence-based 
models that met HHS’s criteria for evidence of effectiveness.® At the same time, states 
can spend part of their MIECHV funding on promising approaches to home visiting as 
long as research is conducted on those promising approaches.® The legislation also 
required an evaluation of MIECHV in its early years, ’° which became the Mother and 
Infant Home Visiting Program Evaluation (MIHOPE). The overarching goal of MIHOPE 
is to provide information about whether families and children benefit from MIECHV- 
funded early childhood home visiting programs, and if so, how. MIHOPE is studying 
these programs as they operated from 2012 to 2017. 


To receive MIECHV funds, awardees were required to create initial plans that 
indicated the communities where the funds would be used and the home visiting models 
that would be supported by those funds.** MIHOPE includes the four evidence-based 
models that 10 states or more chose in their fiscal year 2010-2011 plans for MIECHV 
funding. These four models are: 


e Early Head Start - Home-based option 
e Healthy Families America 
e Nurse-Family Partnership 


e Parents as Teachers 


SHRSA distributes funds from the federal MIECHV Program to MIECHV state and territory award- 
ees. In 2017, HRSA provided awards to 56 states and territories, including 47 state agencies; 3 non- 
profit organizations serving Florida, North Dakota, and Wyoming; the District of Columbia; and 5 U.S. 
territories. Awardees distribute funds to local implementing agencies — also commonly referred to as 
local programs — that work directly with families. Additionally, ACF oversees the tribal MIECHV Pro- 
gram, which as of 2017 funds 29 tribes, consortia of tribes, tribal organizations, and urban Indian 
organizations across 16 states. 

7SEC. 511 [42 U.S.C. 711] (h) (3) (A). 

8SEC. 511 [42 U.S.C. 711] (d) (3) (A) (ii). 

98SEC. 511 [42 U.S.C. 711] (d) (3) (A) (i) (II). 

10SEC. 511 [42 U.S.C. 711] (g) (2). 

“SEC. 511 [42 U.S.C. 711] (e) (1-10). 


MIHOPE includes only women who were pregnant or had children younger than 
6 months of age when they entered the study (a group who are eligible for most 
MIECHV-funded local programs) and who were interested in receiving home visiting 
services. These women were randomly assigned to a MIECHV-funded local home vis- 
iting program or to a control group who received information about other appropriate 
services in the community. MIHOPE estimated the effects of MIECH\V-funded early 
childhood home visiting programs on family and child outcomes around the time children 
were 15 months of age and found positive effects for families across several outcome 
areas that were similar to but somewhat smaller than the average found in past studies 
of individual home visiting models. ‘2 Contact with the MIHOPE families was maintained 
via brief surveys that were completed around the time children were 2.5 and 3.5 years 
of age; this phase of the study is called MIHOPE Check-in. 


Given the positive effects found in previous long-term studies of home visiting 
and previous findings that the benefits of home visiting outweigh the costs only after 
children enter elementary school, '? ACF and HRSA initiated plans to design long-term 
follow-ups with the families who are participating in MIHOPE. Under contract with ACF, 
MDRC is conducting this work in partnership with Columbia University and Mathematica 
Policy Research. This long-term follow-up phase, which focuses on children when they 
are 5 years of age and older, is called MIHOPE-LT. The study design for MIHOPE-LT 
is the subject of this report. 


The primary goal of MIHOPE-LT is to measure the long-term effects of MIECHV- 
funded home visiting programs on family outcomes. The study also plans to examine 
whether the long-term effects of home visiting are larger for some families than for oth- 
ers. In addition, MIHOPE-LT will attempt to determine the pathways through which home 
visiting has long-term effects on families. Finally, another major goal is to conduct a 
benefit-cost analysis that compares the costs of home visiting programs with the benefits 
that the programs generate for participating families, for the government, and for society. 
The four key research questions associated with these goals are discussed further in 
Chapter 4. 


In the interest of providing background and context for the MIHOPE-LT design 
described in this report, the following sections review the MIHOPE design as well as the 
follow-ups and analyses that have already been conducted with the MIHOPE sample. 


2Michalopoulos et al. (2019). 
13See Michalopoulos, Faucetta, Warren, and Mitchell (2017). 


Overview of the Original MIHOPE Design 


MIHOPE included 88 local home visiting programs in 12 states: California, Georgia, Illi- 
nois, lowa, Kansas, Michigan, Nevada, New Jersey, Pennsylvania, South Carolina, 
Washington, and Wisconsin. States were selected based on a number of criteria, includ- 
ing whether they planned to implement more than one of the four evidence-based mod- 
els that MIHOPE is studying and support five eligible local programs or more, whether 
they represented each of four geographic regions in the United States, '* and whether 
they allowed the final sample to include some local programs operating in nonmetropol- 
itan areas. 


The 88 local programs consisted of 19 Early Head Start programs, 26 Healthy 
Families America programs, 22 Nurse-Family Partnership programs, and 21 Parents as 
Teachers programs. As was true for states, local programs had to meet several criteria 
to be included in MIHOPE, such as operating for at least two years when they entered 
the study and being able to recruit enough families to fill program slots and allow for a 
randomly chosen control group. 


MIHOPE Sample 


A total of 4,229 families entered the study from October 2012 to October 2015.5 
The sample includes 1,458 families enrolled through Healthy Families America pro- 
grams, 1,235 enrolled through Nurse-Family Partnership, 963 enrolled through Parents 
as Teachers, and 573 enrolled through Early Head Start — Home-based option. 


To provide reliable estimates of home visiting programs’ effects, women who 
enrolled in the study were randomly assigned to a MIECHV-funded local home visiting 
program or to a control group that received information about other appropriate services 
in the community. To be eligible for MIHOPE, women had to be at least 15 years of 
age, '* be either pregnant or had a child younger than 6 months of age when they en- 
rolled in the study, be able to speak English or Spanish well enough to provide consent 


‘4The major regions were defined using smaller regions defined by HRSA. The four major regions 
used in MIHOPE are the Northeast (HRSA regions 1 to 3), South (HRSA regions 4 and 6), Midwest 
and Plains (HRSA regions 5 and 7), and Mountain and West (HRSA regions 8 to 10). 

‘SOver the course of MIHOPE, 11 families withdrew from the study, 2 sample members from a 
small local program were removed from the analysis, and 1 sample member was found to be too old 
for the study, for a final analytical sample of 4,215 families (2,102 in the program group and 2,113 in 
the control group). 

‘6During its initial review of MIHOPE, the MDRC Institutional Review Board suggested an age 
cutoff because of a concern that women below a certain age would represent a more vulnerable pop- 
ulation than older women. The study team chose 15 years of age based on an estimate that it would 


and complete a survey when they entered the study, and not be receiving home visiting 
services from a participating local program already. They also had to be interested in 
receiving home visiting services and had to meet the relevant local program eligibility 
criteria. 


Women participating in MIHOPE tended to be young, economically disadvan- 
taged, and racially and ethnically diverse, and they exhibited a variety of risk factors at 
study entry that could affect their children’s development.’” Almost two-thirds of the 
women were younger than 25 years of age, and 35 percent were younger than 21 years 
of age. Forty-two percent of the women in the sample did not have high school diplomas; 
as might be expected, older women in the sample were more likely to have completed 
high school. Nearly 75 percent of women in the sample were receiving benefits from the 
Special Supplemental Nutrition Program for Women, Infants, and Children, and more 
than half were enrolled in the Supplemental Nutrition Assistance Program. More than 
half of the women reported that their households had experienced food insecurity in the 
past year, '® nearly one-third reported substance use before pregnancy, over two-fifths 
reported symptoms of either depression or anxiety, and about one-fifth of mothers re- 
ported experiencing or perpetrating physical acts of intimate partner violence. ‘9 


MIHOPE Analyses Conducted with Data Collected Through the First 
Follow-Up Phase 


The first phase of MIHOPE included four main analyses: 


e Implementation research. This analysis had two main goals: (1) to 
describe the services that families received in home visiting programs; 


exclude less than 3 percent of eligible women from the study and because local home visiting pro- 
grams could have had concerns about randomly assigning women younger than that age. As an ad- 
ditional step to protect the rights of women between ages 15 and 18, who were still potentially more 
vulnerable than older women, the study also required a legal guardian to consent to each minor’s 
participation in the study. 

7Michalopoulos et al. (2019). 

‘8Food insecurity was defined as worrying about food or actually running out of food in the year 
before women enrolled in MIHOPE. 

'9As described in Michalopoulos et al. (2019), the MIHOPE sample at study entry was somewhat 
similar to MIECHV enrollees in terms of race and ethnicity, but was younger than the MIECHV enrol- 
lees in fiscal year 2017, which could be because (1) some evidence-based home visiting programs 
enroll women with children who are older than the children in the MIHOPE sample; (2) some MIECHV 
caregivers might not be biological mothers, unlike the MIHOPE sample members; and (3) the age of 
MIECHV enrollees was not measured at the point at which they enrolled in home visiting programs. 
See Michalopoulos et al. (2019) for more information. 


20The first MIHOPE follow-up occurred when children were about 15 months of age. 


and (2) to understand how characteristics of families, home visitors, 
local programs, other home visiting stakeholders (such as the evi- 
dence-based models), and communities are associated with differ- 
ences in the services that families received." 


e Impact analysis. This analysis was used to estimate the effects of 
MIECHV-funded home visiting programs on a broad range of outcome 
areas mentioned in the authorizing legislation and for different sub- 
groups of families using data that were gathered when children were 
about 15 months of age. Effects were estimated in the following out- 
come areas: (1) prenatal, maternal, and newborn health; (2) child 
health and development, including child maltreatment; (3) parenting 
skills; (4) crime or domestic violence; (5) family economic self-suffi- 
ciency; and (6) referrals and service coordination. 2 


e Analysis of impact variation. Specifically, information from the imple- 
mentation study and the impact analysis was used to investigate how 
much effects vary across local home visiting programs, whether any 
features of local programs are associated with larger or smaller effects 
on family outcomes, and whether any aspects of the home visiting ser- 
vices that families received are associated with larger or smaller effects 
on family outcomes.” 


e Cost analysis. The purpose of this analysis is to examine how re- 
sources are allocated at MIECHV-funded home visiting programs, to 
estimate the costs of providing home visiting services to families in 
MIHOPE, and to investigate how the costs vary across families, local 
programs, and evidence-based models. 


Results from the impact and impact variation analyses are summarized in Chap- 
ter 2; detailed information is available in the MIHOPE report that describes effects from 


21Duggan et al. (2018). 

?2Michalopoulos et al. (2019). SEC. 511 [42 U.S.C. 711] (d) (2) (B). The legislation also indicated 
that the program should improve school readiness and academic achievement, but outcomes in that 
area were not included at the first MIHOPE follow-up because children in the sample were 15 months 
of age at that point. 

*3Michalopoulos et al. (2019). 

4Corso, Ingels, and Walcott (forthcoming). 


around the time children were about 15 months of age.”° Findings from the implementa- 
tion analysis are available in the MIHOPE implementation research report.26 Findings 
from the cost analysis will be published in a forthcoming report. 


Limitations of the Study Design 


The use of random assignment and the large number of families and locations 
included in the study provide a strong framework that MIHOPE can use to investigate 
the ability of MIECHV-funded home visiting programs to improve family outcomes, but 
the MIHOPE study design has some important limitations. 


e The team sought to include a diverse set of local home visiting pro- 
grams in the study, but the programs differ in some important respects 
from the larger set of MIECHV-funded programs. For example, 
MIHOPE includes a smaller proportion of rural locations than MIECHV 
as a whole. Thus, the effects presented might differ somewhat from the 
effects of all MIECHV-funded home visiting programs. 


e MIHOPE enrolled local programs and families during the early years of 
the MIECHV Program. Since the implementation of home visiting has 
evolved, the current effects of home visiting may differ somewhat from 
those found for the MIHOPE sample. 


e Although the study strove to collect high-quality information about fam- 
ily outcomes, each data source that was used has some limitations. 
Parent reports may be inaccurate if individuals cannot remember rele- 
vant information or are reluctant to report that information, as may be 
the case with sensitive outcomes. Although administrative data may 
accurately reflect the information collected by state agencies, they are 
limited to families who have not moved from the state and are often 
limited in other respects. For example, Medicaid-reimbursed health 
care is available only for individuals who receive Medicaid benefits. 


e The main results compare outcomes for all families assigned to the 
program group with all families assigned to the control group. Since 
about 17 percent of program group families never received home visits, 
the effects may be larger among families who received services.2’ In 


26Michalopoulos et al. (2019). 

?6-Duggan et al. (2018). 

27See Duggan et al. (2018) for detailed results concerning the amounts of home visiting services 
received by families who enrolled in MIHOPE. 


addition, some control group members received home visits or other 
services to promote positive parenting, and the effects observed in the 
study might have been larger if the control group did not have access 
to such services. 


The Evidence-Based Home Visiting Models Studied in MIHOPE 


In general, home visiting consists of three types of activities: 


e« Assessing family needs. To identify family strengths, needs, con- 
cerns, and interests, home visitors gather information from families 
through formal screening and assessment and through informal means 
that include reading cues provided by family members. 


e Educating and supporting parents. Having identified family needs, 
home visitors devote most of their time to providing education and sup- 
port to families. For example, home visitors educate parents on topics 
such as children’s developmental stages. Home visitors can also pro- 
vide support during crises such as threats of being evicted or incidents 
of family violence. In addition, home visitors work with families to 
strengthen their support networks. Home visitors use methods such as 
positive reinforcement, direct suggestions and encouragement, and 
motivational interviewing to support healthy behavior and positive par- 
enting. 8 


e Referral and coordination. If home visitors think families will benefit 
from receiving more specialized services in the community, they can 
provide referrals to those services. In MIHOPE, referrals were most 
commonly made to address breastfeeding and nutrition, economic self- 
sufficiency, and public assistance or health insurance.2? This aspect of 
home visiting highlights the place of home visiting as one component 
in a comprehensive system of care for early childhood. 


?8Rubak, Sandbeaek, Lauritzen, and Christensen (2005). Motivational interviewing emerged from 
the experiences of clinicians treating individuals with alcohol dependency, and is defined as “a di- 
rective, client-centered counseling style for eliciting behavior change by helping clients to explore and 
resolve ambivalence.” See Miller and Rose (2009). It is viewed as a particularly important technique 
when working with clients who are resistant to changing their behaviors, and when standard cognitive 
behavioral approaches and social learning approaches (that is, positive or constructive reinforcement) 
are not working. See lannos and Antcliff (2013). 


2°Duggan et al. (2018). 


These activities are undertaken to improve outcomes for families in the short 
term. Figure 1.1 shows this intention, as well as how those short-term improvements 
might lead to longer-term benefits for parents and children. 


Although all four evidence-based models use these activities and share the over- 
all goal of improving outcomes for at-risk families and their young children, they differ in 
several important ways. Table 1.1 summarizes some important features of the four evi- 
dence-based models as they existed when MIHOPE began. 


e« Evidence-based model goals. While all four models tried to improve 
child health and development in the broad sense, their specific empha- 
ses differed. For example, Healthy Families America emphasized pre- 
venting child maltreatment; Nurse-Family Partnership strongly empha- 
sized improving maternal and child health; and Early Head Start - 
Home-based option and Parents as Teachers focused on increasing 
positive parenting or school readiness. 


e Intended recipients. Most of these models served families they iden- 
tified as being at risk of poor child outcomes, based on one family char- 
acteristic or more. Although the indicators used to identify families at 
risk differed among the models, most models worked with low-income 
families. Nurse-Family Partnership specifically focused on women 
early in their first pregnancies, while Healthy Families America focused 
on families who faced a variety of risk factors for child maltreatment or 
other negative childhood experiences. Parents as Teachers has histor- 
ically served a broad array of families with children in its target age 
range. All models could enroll women who met the MIHOPE eligibility 
criteria, although Early Head Start and Parents as Teachers also ac- 
cepted families whose youngest children were up to 3 years of age and 
through kindergarten entry, respectively. 


e Other model features. The models also varied somewhat in their pro- 
gram intensity and duration; in the timing, intensity, and content of re- 
quired training; in requirements for group and individual supervision; in 
their standards for home visitor qualifications; and in the degree of flex- 
ibility they allow local programs. 


Home Visiting Activities 


Home visitors 

e Gather information to inform services 

e Provide education and support 

e __ Provide referrals to and coordination 
with needed services 


Home Visiting Activities and Outcomes 


Figure 1.1 


Short-Term Outcomes 


Parents have 


Improved health behaviors during 
pregnancy 

Improved mental health 

Improved economic self-sufficiency 
Reduced levels of intimate partner 
violence 


Parents demonstrate 


Sensitive and competent caregiving for 
infants and children 

Increased use of nonviolent discipline 
techniques 

Reduced harsh parenting practices 
Improved safety of the home 


Newborns and children have 


Better health outcomes 
Age-appropriate development 


Longer-Term Outcomes 


Parents have 

e Improved birth outcomes 

e Improved economic self-sufficiency 
e Improved health and mental health 
e Reduced child maltreatment 


Children have 

e Increased school readiness and 
academic achievement 

e Improved health and mental health 

e Improved life-course outcomes 


SOURCES: Early Head Start Parent, Family, and Community Engagement Framework (https://eclkc.ohs.acf.hhs.gov/sites/default/files/pdf/pfce- 
framework.pdf); Healthy Families America logic model; Nurse-Family Partnership logic model; and Parents as Teachers logic model (https:// 
parentsasteachers.org/logic-model/). The Healthy Families America and Nurse-Family Partnership logic models were obtained from communication with 


the model developers. 
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Selected Planned Services of the Evidence-Based Home Visiting Models in MIHOP 
and Enrollment 


Table 1.1 


E: Goals, Recipients, 


Component 


Evidence-based 
model goals? 


Early Head Start — 
Home-Based Option 


Enhance the development 
of very young children 


Promote healthy family 
functioning 


Promote school readiness 


Healthy Families America 


Build and sustain community 
partnerships to systemically 
engage overburdened parents 
in home visiting services 
prenatally or at birth 


Cultivate and strengthen 
nurturing parent-child 
relationships 


Promote healthy childhood 
growth and development 


Enhance family functioning by 
reducing risk and building 
protective factors 


Prevent child maltreatment and 


adverse experiences 


Nurse-Family Partnership 


Improve prenatal health and 
birth outcomes 


Improve child health and 
development 


Improve families’ economic 
self-sufficiency and maternal 
life course development 


Parents as Teachers 
Provide parents with child 
development knowledge and 
parenting support 

Provide early detection of 
developmental delays and 
health issues 


Prevent child maltreatment 


Increase school readiness 
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(continued) 


Table 1.1 (continued) 


Early Head Start — 


Component Home-Based Option Healthy Families America Nurse-Family Partnership Parents as Teachers 
Intended Low-income pregnant Parents facing challenges such First-time, low-income, pregnant _No eligibility requirements 
recipients women and families with as single parenthood, low mothers and their children for participants 


children from birth to 3 
years of age, families at or 
below the federal poverty 
level, and children with 
disabilities who are eligible 
for Part C services under 
the Individuals with 
Disabilities Education Act 
in their states 


incomes, childhood histories of 
abuse or adverse experiences, 
current or past behavioral health 
issues, or domestic violence 


Local programs select the specific 
characteristics of the target 
populations they plan to serve 


Local programs select the 
specific characteristics of 
their target populations, 
such as children with special 
needs, families at risk for 
child abuse, low-income 
families, teen parents, first- 
time parents, immigrant 
families, families with little 
literacy, or parents with 
mental health or substance 
use issues 


Intended timing 
of enrollment 


Pregnancy through age 3 


Before the end of the 28th 
week of pregnancy? 


Pregnancy or within the first 3 
months after a child’s birth 


Pregnancy or soon after 
birth, though can continue 
until age 5 


SOURCES: Evidence-based model websites (Early Head Start: https://eclkc.ohs.acf.hhs.gov/programs/article/early-head-start-programs; Healthy Families 
America: www.healthyfamiliesamerica.org; Nurse-Family Partnership: www.nursefamilypartnership.org; Parents as Teachers: parentsasteachers.org), the 
U.S. Department of Health and Human Services Home Visiting Evidence of Effectiveness (HomVEE) website (homvee.acf.hhs.gov/programs.aspx), and 

interviews the study team conducted with the MIHOPE evidence-based model developers. 


NOTES: #Goals are as stated by each evidence-based model. 
bLocal programs are recommended to begin conducting visits as early as possible in the pregnancy. 
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Checking in with Families When Children Are Preschool Age 


As noted earlier, in addition to following up with the MIHOPE families when the study 
child (who is also referred to as “the child” or “the MIHOPE child” in the remainder of the 
report) was 15 months of age, follow-up data collection has also been conducted (1) 
when the child is 2.5 years of age, and (2) when the child is 3.5 years of age. The phase 
of MIHOPE that includes these two later follow-up points is called MIHOPE Check-in, 
and data collection for this phase began in September 2015 and concluded in June 
2019.%° 


MIHOPE Check-in included brief surveys to gather information from parents on 
the topics shown in Table 1.2. Information about these outcomes will allow the study to 
estimate ongoing effects of home visiting as children grow older. Updated contact infor- 
mation was also obtained at each point in preparation for the MIHOPE long-term follow- 
up. Because the data were still being collected when the long-term follow-up study was 
being designed, they did not contribute to the MIHOPE-LT design. 


Context and Motivation for MIHOPE-LT 


As Chapter 3 discusses, several previous studies have provided information on the long- 
term effects of home visiting programs. MIHOPE-LT can expand this body of evidence. 
The previous studies had relatively small samples (most included fewer than 1,000 fam- 
ilies), were model-specific, and did not all examine the same outcomes in the same way 
across models, making it difficult to summarize across studies and models. In contrast, 
MIHOPE-LT will measure the same outcomes for all four evidence-based models. In 
addition, most of the previous long-term studies were completed many years ago.*' 
Home visiting programs have changed over time, both because of statutory require- 
ments to meet the federal benchmarks tied to MIECHV funding and because programs 


%°Data collection for the survey when children are about 2.5 years of age began in September 
2015 and concluded in June 2018. Data collection for the survey when children are about 3.5 years 
of age began in June 2017 and concluded in June 2019. 

31Most of the studies began enrolling families before 1995, and most follow-ups occurred before 
2005. 
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Table 1.2 
MIHOPE Check-in Constructs at 2.5 and 3.5 Years 


Survey Outcome Area and Construct 


2.5- and 3.5-year survey Maternal health 
Self-reported health status 
Subsequent pregnancies and births 
Depressive symptoms 


Family economic self-sufficiency 
Child care 


Education or training 
Employment 
Public assistance receipt? 


Parenting 

Discipline 

Home literacy environment 
Cognitive stimulation 


Child health 

Health status 

Well-child visits 

Has primary care provider 
Emergency department visits 
Has health insurance coverage 


Participation in home visiting or parenting program 


3.5-year survey only Child development 
Early language and literacy skills 
Early math skills 
Fine motor skills 
Executive function 


NOTE: *The benefits that were measured were those from the Supplemental Nutrition As- 
sistance Program; disability insurance; Temporary Assistance for Needy Families; and 
Special Supplemental Nutrition Program for Women, Infants, and Children. 


continually evolve.32 Moreover, the context in which the programs operate, and the pro- 
gram participants, have also changed. As programs evolve and program context 
changes, additional evaluation can determine whether programs continue to be effective 
in meeting their goals. 


32SEC. 511 [42 U.S.C. 711] (d) (1) (A) (i-vi) shows the federal benchmark areas under the 
MIECHV Program. 
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Contents of This Report 


The subsequent chapters in this report provide the following information: 


Chapter 2, Early Findings from MIHOPE on Effects for Families, presents results 
from the first follow-up with MIHOPE families, which occurred around the time children 
were 15 months of age. 


Chapter 3, Long-Term Effects of Home Visiting: Evidence from Previous Studies, 
summarizes evidence from existing long-term studies of the four MIHOPE home visiting 
models, on the effects of home visiting and benefit-cost analyses. 


Chapter 4, Overview of Plans for MIHOPE Long-Term Follow-Up, presents the re- 
search questions that MIHOPE-LT is designed to answer, the proposed timing of poten- 
tial follow-up points, the sampling the study may use, and the data sources and outcome 
areas that may be examined. 


Chapter 5, The MIHOPE-LT Kindergarten Follow-Up, describes the first MIHOPE-LT 
data collection effort, which began in January 2019 and is occurring approximately five 
years after women enrolled in MIHOPE, when their children are kindergarten age.*3 This 
chapter describes plans for the kindergarten data collection effort, focusing on the con- 
structs to be examined. 


Chapter 6, Conclusion, highlights the contributions that MIHOPE-LT is expected to 
make to the research evidence on home visiting and discusses challenges the study 
could face in collecting data over many years as well as ways these challenges can be 
addressed. 


33MIHOPE children were either infants (under 6 months of age) or in utero when their mothers 
entered the study. 
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Chapter 2 


Early Findings from MIHOPE on Effects for Families 


This chapter describes results from the MIHOPE follow-up that occurred around the time 
children were 15 months of age. Results are presented that address two broad ques- 
tions: 


1. What are the effects of home visiting programs across the range of outcomes 
specified in the authorizing legislation? 


2. Are the effects of home visiting larger among some types of families than for 
others? 


Data Sources 


Around the time the MIHOPE child was 15 months of age, the study team collected 
information about family and child outcomes from several data sources, including: 


e A one-hour telephone interview with the child’s mother,‘ during which 
she was asked about outcomes in all the domains mentioned in the 
authorizing legislation other than school readiness and academic 
achievement 


e Avvideo recording of an interaction between the child and mother using 
the “Three Bags” and “Clean-Up” tasks, during which the child and 
mother played with toys contained in three bags and placed the toys 
back in the bags 


e The Preschool Language Scales, Fifth Edition, Auditory Comprehen- 
sion scale, to assess the child’s ability to be attentive and respond to 
stimuli in the environment and to comprehend basic vocabulary or ges- 
tures 


e Ameasurement of the child’s weight and height, to provide information 
about whether the child’s growth was within a normal range or the child 


‘In 64 cases where the mother was not available to answer the survey (in most cases because 
she no longer had custody of the child), data collection was conducted with the child’s primary care- 
giver. 
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exhibited early signs of being underweight or at risk of becoming over- 
weight; and a measurement of the mother’s weight to assess the ef- 
fects of home visiting on maternal weight and obesity 


e The Infant-Toddler Home Observation for Measurement of the Envi- 
ronment, to measure the quality and amount of stimulation the child 
could receive in the home using observations from study team data 
collectors in the family’s home and parent responses to the 15-month 
survey 


e Administrative data (data collected while administering a public pro- 
gram) in three areas: (1) health care use (for which data came from 
Medicaid and the Children’s Health Insurance Program), (2) child mal- 
treatment (for which data came from state administrative child welfare 
records), and (3) employment and earnings (for which data came from 
the federal National Directory of New Hires) 


Outcomes 


Following best practices in a random assignment study, MIHOPE estimated the effects 
of evidence-based early childhood home visiting programs by comparing outcomes for 
all families assigned to the program group with all families assigned to the control group. 
Results included most outcome areas that the legislation that authorized the Maternal, 
Infant, and Early Childhood Home Visiting (MIECHV) Program indicated the program 
should affect, including (1) prenatal, maternal, and newborn health; (2) child health and 
development, including child maltreatment; (3) parenting skills; (4) crime or domestic 
violence; (5) family economic self-sufficiency; and (6) referrals and service coordination. 2 
To focus the analysis on areas where home visiting programs were likely to have their 
greatest short-term effects, the study team chose 12 outcomes based on the evidence 
of effects from the four evidence-based models included in MIHOPE (Early Head Start 
— Home-based option, Healthy Families America, Nurse-Family Partnership, and Par- 
ents as Teachers) that existed before the analysis began. The team also considered the 
policy relevance of those outcomes and the quality of the tools available to measure the 


2SEC. 511 [42 U.S.C. 711] (d) (2) (B). The legislation also indicated that the program should 
improve school readiness and academic achievement, but children in MIHOPE were too young 
to provide information about that area at the follow-up that occurred when they were 15 months 
of age. 
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outcomes. Following the terminology used in a report written for the Institute of Educa- 
tion Sciences, the 12 outcomes, which are listed in Table 2.1, are considered “confirm- 
atory.”3 


MIHOPE also estimated effects for several dozen “exploratory outcomes” that 
capture other aspects of the areas the legislation intended home visiting to improve. 
These outcomes were considered exploratory because past home visiting studies had 
not found effects on them or they had not been examined in previous studies. They 
were still thought to be areas where MIECHV-funded programs might improve family 
outcomes. These outcomes are listed in Table 2.2. 


Table 2.1 
MIHOPE Confirmatory Outcomes at 15 Months of Age 


Outcome Area 


Outcome 


Maternal health 
Family economic self-sufficiency 


Parenting 


Child maltreatment 


Child health 


Child development 


New pregnancy after study entry 
Receiving education or training 


Quality of the home environment 
Parental supportiveness 


Frequency of minor physical assault during the past year 
Frequency of psychological aggression during the past year 


Health insurance coverage for the child 

Number of Medicaid-paid well-child visits 

Number of Medicaid-paid emergency department visits 

Any Medicaid-paid health care encounter for injury or ingestion 


Behavior problems 
Receptive language skills 


Differences between the program and control group outcomes were also esti- 
mated for seven subgroups that were created based on the following demographic char- 
acteristics and psychosocial risk factors, which were measured when women enrolled 


in the study: 


e Gestational age (number of weeks pregnant at enrollment) or whether 
the woman had already given birth 


3Schochet (2008). 


18 


Table 2.2 


MIHOPE Exploratory Outcomes at 15 Months of Age 


Outcome Area Outcome 
Maternal health e Health insurance coverage for the mother 
e Current smoking 
e Substance use during the past three months 
e Depressive symptoms 
e Health status self-rated as “poor” or “fair” 
e Received any behavioral health services 


Family economic self-sufficiency 


Received any public assistance during the past month 
o Supplemental Nutrition Assistance Program 
o Disability insurance 
o Temporary Assistance for Needy Families 
o Women, Infants, and Children 
Food insecurity 
Employed five quarters after birth 
Earnings five quarters after birth 
Use of nonparental child care 
Received any transportation services 


Intimate partner violence 


Maternal perpetration of physical violence 

Maternal experience with physical or sexual violence 
Experience with battering 

Received any domestic violence services 

Received any services from a domestic violence shel- 
ter 


Parenting 


Quality of the home environment 

o Parental warmth 

o Parental support for learning and literacy 

o Parental verbal skills 

o Parental lack of hostility 

o Home interior 
Parental supportiveness 

o Sensitivity 

o Positive regard 

o Stimulation of cognitive development 
Parental unsupportiveness 

o Intrusiveness 

o Negative regard 

o Detachment 
Parental discipline 

o Discipline strategies during parent-directed task 

= Gentle guidance 
= Control 

o Nonviolent discipline during the past year 
Parenting stress 

o Parenting distress 

o Parent-child dysfunctional interaction 
Awareness of health and safety hazards 


(continued) 
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Table 2.2 (continued) 


Outcome Area Outcome 


Child maltreatment Severe or very severe physical abuse during the past year 
Any substantiated maltreatment report 
Any maltreatment report 


Loss of custody 


Child health e Primary care provider for the child 
e Number of Medicaid-paid immunizations 
e Any nonbirth hospitalizations 
e Weight for length 
o Underweight 
o Normal weight 
o Atrisk of overweight 
e Duration of breastfeeding 


Child development e Social-emotional competence 

e Behavior during semistructured play 
o Engagement of the parent 
o Negativity toward the parent 
o Sustained attention to objects 

e Behavior during a parent-directed task 
o Compliance 
o Distress 

e Received any early intervention services 


e Parity (whether the woman had children before she entered the study, 
not counting her pregnancy or her newborn) 


e The mother’s race and ethnicity 
e The presence of intimate partner violence 


e The mother’s level of emotional functioning in three areas: the pres- 
ence of depression, relationship anxiety, and relationship avoidance at 
the time she entered the study‘ 


e The mother’s psychological resources® 


4“Relationship anxiety” refers to an individual’s excessive need for reassurance, fear of rejection, 
and a desire to merge with relationship partners. “Relationship avoidance’ reflects the extent to which 
an individual avoids intimacy and is distrusting of others. See McFarlane et al. (2010); Mikulincer and 
Shaver (2007). 

5The concept of “psychological resources” is taken from the Nurse-Family Partnership Memphis 
pilot test, which hypothesized that effects on maternal caregiving and childhood injuries would be 
greater among mothers with few psychological resources. See Kitzman et al. (1997). In MIHOPE, it is 
based on a composite of (1) mental health, (2) mastery (the extent to which a person thinks life 
chances are under her control), and (3) verbal abstract reasoning. 


20 


e Demographic characteristics of mothers that put them or their children 
at higher risk of poor outcomesé® 


Impact Findings When Children Are 15 Months of Age 


Figure 2.1 displays estimated effects on the 15-month confirmatory outcomes. (See Box 
2.1 for an explanation of how to interpret estimated effects.) Effects on confirmatory and 
exploratory outcomes and across subgroups are summarized below. 


e There are positive effects for families in MIHOPE. Most estimated 
effects are similar to but somewhat smaller than the average 
found in past studies of individual home visiting models. 


Estimated effects are statistically significant for 4 of the 12 confirmatory out- 
comes:’ the quality of the home environment, the frequency of psychological aggression 
toward the child, the number of Medicaid-paid child emergency department visits, and 
child behavior problems. Overall, for 9 of the 12 confirmatory outcomes, program group 
families fared better than control group families on average,® which is unlikely to have 
occurred for the study sample if the home visiting programs made no true difference in 


Risk factors included the following: mothers’ receipt of public assistance or enrollment in Medi- 
caid; mother was 20 years of age or younger; child’s biological father did not live in the home; and 
mother was not enrolled in school (if younger than 19 years of age) or had not received a high school 
degree (if at least 19 years of age). 

7The p-value indicates the likelihood of estimating an effect of this magnitude or larger in absolute 
value if the intervention had zero effect (that is, if the estimated effect had occurred by chance). In 
Michalopoulos et al. (2019), estimates are considered statistically significant if there is less than a 10 
percent likelihood that the effect is due to chance based on a two-tailed t-test (which assumes effects 
have the possibility of appearing in a positive or negative direction); that is, if the p-value is less than 
0.1. 

8This tally includes five outcomes where program group families had better outcomes on average 
than control group families but where the difference between them is not statistically significant. 
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Figure 2.1 
Estimated Effects on Confirmatory Outcomes at 15 Months of Age 


Outcomes where positive effects mean improvements for families 


-0.2 -0.1 0.0 0.1 0.2 


Receiving education or training 

Quality of the home environment —_o—_- 
Parental supportiveness 
Health insurance coverage for the child 
Number of Medicaid-paid well-child visits 


Receptive language skills 


Outcomes where negative effects mean improvements for families 


-0.2 -0.1 0.0 0.1 0.2 


New pregnancy after study entry 
Frequency of minor physical assault during the past year 
Frequency of psychological aggression during the past year —_o— 
Number of Medicaid-paid child emergency department visits —__o—_ 
Any Medicaid-paid health care encounter for injury or ingestion 


Behavior problems —e— 


® Effect size == Confidence interval (90 percent) 


SOURCES: Calculations based on the MIHOPE 15-month follow-up survey, the in-home assessment, the mother-child 
video-recorded interaction, and Medicaid enrollment and claims data. 


NOTES: Effects are considered statistically significant if the 90 percent confidence interval does not intersect with 0. A 


statistical test of the number of outcomes for which estimated effects would be improvements for families resulted in a 
p-value of 0.096 for having 9 or more such findings out of 12. 
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Box 2.1 


How to Interpret Estimated Effects 


The effects of home visiting are estimated by comparing the outcomes of the program 
and control groups, adjusted for background characteristics of the sample members. 
Figure 2.1 shows the estimated effects for the study’s confirmatory outcomes as circles. 
For example, there is a small, negative estimated effect on whether a child had health 
insurance coverage at 15 months but a small, positive estimated effect on whether a 
mother was receiving education or training at 15 months. 


All results are presented as effect sizes, which is a way of standardizing outcomes so 
they are on the same scale. The interpretation of an effect size will vary with the out- 
come and the context, so it is difficult to characterize the magnitude of effect sizes in 
general. A standard IQ test has a standard deviation of 10, for example, so an effect 
size of 0.10 would represent a one-point positive change in IQ. For an outcome ex- 
pressed as a percentage, such as the percentage of mothers with a subsequent preg- 
nancy, an effect size of 0.10 would represent a change of about 3 percentage points to 
5 percentage points in the outcome. 


The lines surrounding the estimated effect in Figure 2.1 represent the 90 percent con- 
fidence interval, an estimate of the variability (or statistical imprecision) of the effects. A 
narrower confidence interval suggests a more precise estimate than a wider confidence 
interval; a wider interval indicates greater variability and thus greater uncertainty. Con- 
fidence intervals that do not contain zero (that is, that are fully to the right or the left of 
the zero line in the figure) indicate that the effect is different from zero to a statistically 
significant degree, using 10 percent as the benchmark for statistical significance. That 
is, there is less than a 10 percent chance of finding an estimated effect this big if the 
true effect of the program were zero. The figure shows that the effect is different from 
zero to a statistically significant degree for four outcomes: quality of the home environ- 
ment, frequency of psychological aggression toward the child during the past year, 
number of Medicaid-paid child emergency department visits, and child behavior prob- 
lems. 
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family outcomes.® However, no outcome or outcome area stands out as having consist- 
ently large effects.'° In addition, the effects are generally smaller than those found in 
past studies, although MIHOPE differs from those studies in many respects. For exam- 
ple, most of those studies were conducted in a single local area rather than including 
sites across the country, and some were conducted many years ago, when similar ser- 
vices were less likely to be available to control group families. In addition, previous stud- 
ies each examined only one evidence-based model, and might have chosen outcomes 
where those models were expected to make the largest differences. 


e« There are some statistically significant differences in effects on 
the confirmatory outcomes among the evidence-based models 
that are generally consistent with the models’ focuses. 


For example, Parents as Teachers produced the largest increase in parental 
supportiveness, and Nurse-Family Partnership produced the largest reduction in emer- 
gency department visits for children. The differences are somewhat sensitive to the sta- 
tistical method used to examine them but these two patterns were found across different 
estimation methods. 


e« Most estimated effects are not statistically significant. 


Although the results suggest that families are benefiting from MIECHV-funded 
home visiting services, only about one-third of the confirmatory outcomes and one-third 
of the exploratory outcomes showed effects that were statistically significant. In addition, 
only one of the 67 estimated effect sizes is greater than 0.20, a level sometimes used 
as a threshold for considering an effect to be small." 


e Results for several exploratory outcomes suggest home visiting 
may improve maternal health. 


°A statistical test of the number of outcomes for which estimated effects would be positive resulted 
in a p-value of 0.096 for having 9 positive findings or more out of 12, meaning there is less than a 10 
percent probability that this pattern of results would have resulted if home visiting had no effect on any 
of the 12 outcomes. A statistical test that accounts for the magnitude of the estimated effects has a p- 
value of 0.025, meaning there is a 2.5 percent probability that this pattern of results would have been 
found if home visiting had no effects on the 12 outcomes. 

'°ln addition, after adjusting for the number of confirmatory outcomes, none of the 12 estimated 
effects is statistically significant. Although the evidence as a whole points to positive effects for families, 
this finding reduces the study team’s confidence that any individual outcome was improved by the 
home visiting services that were studied. 


Cohen (1988). 
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MIHOPE found statistically significant improvements in women’s general health, 
increases in health insurance coverage, and reductions in depressive symptoms (al- 
though program group mothers were also more likely to say they had used drugs or 
alcohol in the recent past). Results for exploratory outcomes are not shown in Figure 
2.1.'2 Improving maternal mental health could be especially important since it could re- 
sult in improvements in many other areas, such as child development and economic 
self-sufficiency. 


e Home visiting might reduce household aggression. 


The results also suggest that home visiting services reduce household aggres- 
sion, which could have wide-ranging, long-term implications. For example, there are sta- 
tistically significant effects on the frequency of psychological aggression toward children 
(a confirmatory outcome) as well as mothers’ experience with intimate partner violence 
and mothers’ use of domestic violence services (exploratory outcomes). This effect is 
consistent with other significant effects — for example, those on exploratory outcomes 
such as parental depression (discussed above), parental stress, and parental discipline 
using gentle guidance. Reduced household aggression and improved parenting behav- 
iors could also help explain observed reductions in child behavior problems (a confirm- 
atory outcome). Because adverse childhood experiences such as child abuse and inti- 
mate partner violence have been shown to be associated with negative long-term 
outcomes, reducing household aggression could benefit children as they grow older. 


e Differences in estimated effects for the 12 confirmatory outcomes 
across subgroups of families are generally small and not statistically 
significant. 


Of the 84 comparisons of effects that were made to examine effects for sub- 
groups of families, only 8 differences were statistically significant at the 10 percent level. 
This pattern would be expected to occur by chance even if there were no real differences 
in effects across subgroups. Moreover, after applying an adjustment for conducting mul- 
tiple tests, the only statistically significant difference in estimated effects is by race and 
ethnicity for a single outcome: the number of Medicaid-paid well-child visits. 


12See Michalopoulos et al. (2019). 

'3Felitti et al. (1998). 

‘4Specifically, the children of non-Hispanic white mothers in the program group had significantly 
fewer well-child visits than children of non-Hispanic white mothers in the control group. The estimated 
effects for the other subgroups were positive but small (children of mothers in the program group had 
more Medicaid-paid well-child visits than children of mothers in the control group). 
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Summary 


MIHOPE estimated the effects of MIECHV-funded early childhood home visiting pro- 
grams on family and child outcomes around the time children were 15 months of age 
and found effects for families across several outcome areas that were generally similar 
to but somewhat smaller than the average found in past studies of the individual home 
visiting models. *® Subsequent chapters describe the design for long-term follow-up with 
MIHOPE families. 


‘SMichalopoulos et al. (2019). 
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Chapter 3 


Long-Term Effects of Home Visiting: Evidence from 
Previous Studies 


This chapter provides an overview of both the evidence on the effects of home visiting 
and the evidence generated by benefit-cost analyses from these earlier studies to situate 
MIHOPE-LT within this existing body of research. ' 


Evidence of Effectiveness 


Evidence on the effects of home visiting from previous studies of the models included in 
MIHOPE (Early Head Start - Home-based option, Healthy Families America, Nurse- 
Family Partnership, and Parents as Teachers) informed the MIHOPE-LT design by 
providing information about previous results that could potentially be replicated and gaps 
in current knowledge that needed to be filled.2 Because MIHOPE-LT is following up with 
families around the time the children are 5 years of age — that is, around the time they 
start kindergarten — this section of the chapter covers follow-ups in previous studies 
that occurred when children were 5 to 21 years of age. 


As indicated in Table 3.1, all the current available evidence comes from seven 
model-specific studies. All but one of the studies was a randomized controlled trial.4 The 
current evidence for children over 11 years of age comes only from studies of Nurse- 
Family Partnership. 


Table 3.2 presents the evidence from these studies. In each study, families were 
enrolled when children were infants or their mothers were pregnant, so the table shows 
study results as the children age. 


‘See, also, Michalopoulos, Faucetta, Warren, and Mitchell (2017). 

2With one exception, the studies mentioned in this report are those that are listed on the Home 
Visiting Evidence of Effectiveness (HomVEE) website (http://nomevee.acf.hhs.gov) as being of either 
high quality or moderate quality. The fifth-grade follow-up of the national Early Head Start evaluation 
is not listed on the HomVEE site. In addition, the four models’ developers confirmed that there were 
no other published studies with long-term impact findings. Finally, existing studies of Parents as 
Teachers with children older than 5 years of age are not considered to be of high or moderate quality 
by the HomVEE review and are not included in this report. 

3Drazen and Haust (1993) had a prekindergarten follow-up that included some children who were 
4 years of age and some children who were 5 years of age, so it is included here. However, any 
studies that looked only at children 4 years of age and younger were excluded. 

4The one exception is the Parents as Teachers study. 
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Table 3.1 


Seven Studies with Follow-Ups at Ages 5 to 21 Years 
That Informed the MIHOPE-LT Design 


Baseline Sample Size 


Home Visiting Model Study Locale (Number of Families) Follow-Up Points 
Early Head Start — Nationwide 1,3852 Kindergarten 
Home-based option Fifth grade 
Healthy Families Erie, Rensselaer, and 1,173 First grade 
America Ulster counties, New York 

Oahu, Hawaii 6845 Averaged outcomes 


over 7-9 years old 


Nurse-Family Elmira, New York 400 15 years old 
Partnership 19 years old 


Denver, Colorado 735 6 years old 
9 years old 


Memphis, Tennessee 1,139 6 years old 
9 years old 
12 years old 
20-21 years old 


Parents as Teachers Binghamton, New York 40 4-5 years old° 


SOURCES: Bair-Merritt et al. (2010); Chazan-Cohen, Raikes, and Vogel (2013); Drazen and Haust (1993); 
DuMont et al. (2010); Eckenrode et al. (2001); Eckenrode et al. (2010); Hanks et al. (2011); Holmberg, 
Luckey, and Olds (2011); Jones Harden, Chazan-Cohen, Raikes, and Vogel (2012); Kirkland and Mitchell- 
Herzfeld (2012); Kitzman et al. (2010); Olds et al. (1997); Olds et al. (1998); Olds et al. (2007); Olds et al. 
(2010); Olds, Holmberg et al. (2014); Olds, Kitzman et al. (2004); Olds, Kitzman et al. (2014); Sidora-Arcoleo 
et al. (2010); Vogel et al. (2010); and Zielinski, Eckenrode, and Olds (2009). 


NOTES: With one exception, the studies mentioned in this table are those that are listed on the Home Visit- 
ing Evidence of Effectiveness (HomVEE) website (http://nhomevee.acf.hhs.gov) as being either high quality 
or moderate quality. The fifth-grade follow-up of the national Early Head Start (EHS) evaluation is not listed 
on the HomVEE site. In addition, the four models’ developers confirmed that there were no other published 
studies with long-term impact findings. Finally, existing studies of Parents as Teachers with children older 
than 5 years of age are not considered to be of high or moderate quality by the HomVEE review and are not 
included in this report. 

aThis is the sample size for participants who were only enrolled in home-based EHS programs. The na- 
tional EHS evaluation sample size is larger as it also includes participants enrolled in center-based and 
mixed-approach programs. 

bSeven hundred and thirty families gave initial consent to participate in the study, but only 684 were inter- 
viewed at baseline. Forty-one of these families were assigned to a smaller “testing control group,” which is 
why some studies cite a sample size of 643. 

°A study (Drazen and Haust, 1993) with a prekindergarten follow-up included some children who were 4 
years of age and some children who were 5 years of age, so it is included here. However, any studies that 
looked at only children age 4 years and younger were excluded. 
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Table 3.2 


Evidence from Previous Home Visiting Studies (Follow-Ups at Ages 5 to 21) 


Age Group 
5 Years and 
Outcome Area Kindergarten 6 Years 7 Years 7-9 Years 9 Years 5" Grade 12 Years 15 Years 19-21 Years 
Child Development PPPPPPPPPP NNNNNN - HHHHHHH NNNNNN = EEEEEE NNNNNN N N 
BPRISERCAl PPPPPPPPPPP NNNNNN HHHHHHH NNNNNN ~~ EEEEEE NNNNN 
BerOmiance EEEEEEEEEE NNNNN HHHHHHH NNNNNN ~~ EEEEEE NNN 
EEEE HHHHHHH NNNNNN ~ EEEEEE 
Family Economic P NNNNN NNNNNN EEEEEEE NNNNNN-~ NNNNN NNNN 
Self-Sufficiency EEE NNNNN NNNNNNN NNNNNNN 
NNNN NNNNNN NNNNNNN 
Juvenile Delinquency, [3 N HHHHHH ~—= NNN EE NNN NNNNNNN NNNNNN 
Family Violence, and HHHH NNNNNNN NNNNNNN 
Crime NNNNNNN- NN 
EE NNNNN NNNNN EEE NNNNN NNNN NNN 
Maternal Health NNNNN NNNNN NN 
PPP HHHHHHHH N NNNN 
Child Maltreatment E HHHHHHHH NNN 
HHHHHHHH 
NN HHH NNN EEEE NNN NNNNN NNNNNNN 
Child Health NNN NNN 
EEEEEEE HHHH EEEEE 
EEEEE 
EEE 


SOURCES: The U.S. Department of Health and Human Services Home Visiting Evidence of Effectiveness (HomeVEE) website (homvee.acf.hhs.gov) and 
sources in Table 3.1. 


NOTES: E = Early Head Start H = Healthy Families America N = Nurse-Family Partnership P = Parents as Teachers 


Bold = Outcome examined, significant. NO BOLD = Outcome examined, not significant. Red indicates the result was negatively significant. 
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Of the 405 total outcomes examined in previous studies, 68 estimates (17 per- 
cent of examinations) are statistically significant and positive,® which is more than would 
be expected if home visiting had no effect on these outcomes. Across this large set of 
findings, about 5 percent would be statistically significant at the 5 percent significance 
level even if home visiting had no benefits for families as children age.°® 


The likelihood that estimates are statistically significant varies across domains, 
but more than 5 percent of the estimates are statistically significant in a favorable direc- 
tion in all of them. Family economic self-sufficiency (25 percent), parenting (25 percent), 
and maternal health (21 percent) have the highest rates of statistically significant positive 
results. Child development and school performance (14 percent), child health (12 per- 
cent), child maltreatment (14 percent), and juvenile delinquency, family violence, and 
crime (13 percent) are at the lower end of the range. 


Following is a summary of the overall evidence from each outcome area shown 
in Table 3.2 (in descending order from most to least examined): 


e Child development and school performance (144 examinations; 14 
percent statistically significant): Positive effects have been concen- 
trated in follow-ups at younger ages, for which 80 examinations have 
been made and 19 statistically significant estimates found; after age 7, 
there have been 64 examinations in this domain and only one statisti- 
cally significant estimate. 


e« Family economic self-sufficiency (73 examinations; 25 percent sta- 
tistically significant): The positive effects have been found in increased 
parental employment, reduced receipt of benefits, and increased family 
stability through age 12. 


e Juvenile delinquency, family violence, and crime (maternal) (56 
examinations; 13 percent statistically significant overall) 


o Juvenile delinquency (32 examinations; 22 percent statistically 
significant): The few positive effects are seen on arrests and con- 
victions at ages 15 and 19 (in the Nurse-Family Partnership Elmira 


5In this case, estimates are considered statistically significant if there is less than a 5 percent 
likelihood that the effect is due to chance based on a two-tailed t-test (that is, assuming effects have 
the possibility of appearing in a positive or negative direction). 

SAlthough statistical significance is not a perfect indicator of the size of the impacts, estimated 
impacts that are not statistically significant in individual studies are generally quite small. 
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follow-ups), which suggests that these outcomes should be exam- 
ined in middle or late adolescence. 


o Family violence (16 examinations; 0 percent statistically signifi- 
cant): Most examinations come from the Healthy Families Amer- 
ica Hawaii follow-up (at ages 7 to 9 years). 


o Crime (maternal) (8 examinations; 0 percent statistically signifi- 
cant): The crime outcomes that were examined for mothers were 
jail, arrests, and convictions (at the Nurse-Family Partnership fol- 
low-ups at ages 9, 12, and 15 years). 


e Maternal health (39 examinations; 21 percent statistically significant): 
The evidence suggests positive effects are more likely to be found on 
the mother’s subsequent births and pregnancies before the child for 
whom the mother entered home visiting is 9 years of age. Effects have 
also been found on mothers’ sense of mastery.’ 


e Child maltreatment (36 examinations; 14 percent statistically signifi- 
cant): The majority of examinations of child maltreatment come from 
the 7-year Healthy Families America follow-up and the 15-year Nurse- 
Family Partnership follow-up. Effects have been seen on parents’ re- 
ports of minor physical aggression and reports of child abuse and ne- 
glect substantiated by child protective services agencies. 


e Child health (33 examinations; 12 percent statistically significant and 
positive): Examinations of child health are concentrated in the areas of 
adolescent substance use and reproductive behaviors. Significant ef- 
fects on substance use occur at the 12-year Nurse-Family Partnership 
follow-up but not at later follow-ups, and no significant effects on repro- 
ductive behaviors have been observed. 


e Parenting (24 examinations; 25 percent statistically significant): Par- 
enting outcomes have been examined at only three follow-up points, 
and the majority of examinations occurred at the two Early Head Start 
follow-up points. The significant results are concentrated at the Early 
Head Start kindergarten follow-up; no impacts were observed on any 
of the parenting outcomes that were examined at the fifth grade follow- 


up. 


7Mastery measures the extent to which a person thinks life chances are under her control. 
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Discussion about how the evidence from previous studies informs data collection 
plans for the MIHOPE-LT kindergarten follow-up is included in Chapter 5. 


Benefit-Cost Evidence 


In addition to examining the effectiveness of home visiting, previous studies have exam- 
ined whether the monetary benefits to families, the government, and society outweigh 
the costs of providing home visiting services.? In these analyses, benefits are often 
based on outcomes such as earnings and receipt of public assistance benefits that are 
already expressed in monetary terms. In addition, they can be based on outcomes 
where there is useful information that links the outcome to monetary benefits, such as 
educational attainment (which can contribute to increased earnings and reduced use of 
public assistance programs) and child maltreatment (which may reduce costs to the 
child welfare system). Since it can be difficult to link outcomes such as improved parent- 
ing practices to monetary benefits, the benefits of home visiting that are estimated in 
these analyses might understate their full value to individuals, the government, or soci- 
ety.° 


Table 3.3 summarizes results from benefit-cost analyses of the evidence-based 
models and includes results from several studies. 1° 


e Healthy Families America: Two studies, one of 2,727 families in Or- 
egon with one follow-up that occurred when children were 2 years of 
age, and a second of 1,173 families in New York with follow-ups that 
occurred until children were 7 years of age; plus a Washington State 
Institute for Public Policy (WSIPP) analysis that made several assump- 
tions to project from specific follow-up periods to the parent's and 
child’s lifetimes. 


e Nurse-Family Partnership: Three studies, in Denver (735 families), 
Elmira (400 families), and Memphis (1,139 families), with multiple fol- 
low-up points, and one WSIPP lifetime projection analysis. 


8Results in this section are based on Michalopoulos, Faucetta, Warren, and Mitchell (2017). How- 
ever, the Washington State Institute for Public Policy (WSIPP) benefit-cost ratios reported in that study 
are higher than those in this report because WSIPP periodically changes its estimates based on up- 
dated benefit-cost methods and literature reviews. This report uses the WSIPP results available in 
2018. 

°Because different studies may not have included the same outcomes in estimating the benefits 
of home visiting, there may be differences in benefit-cost results across studies. 


10A benefit-cost analysis has not been published for Early Head Start. 
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Table 3.3 


Directly Measured and Lifetime Projections of Returns on Investment in Home 
Visiting, by Evidence-Based Model 


Model and Study 


Healthy Families America 
Oregon: Green et al. (2016) 


New York: DuMont et al. (2010) 
Oregon: Green et al. (2016) 
WSIPP (2018a) 


Nurse-Family Partnership 
Elmira: Olds et al. (1993) 


Denver: Glazner et al. (2004) 
Memphis: Glazner et al. (2004) 
Denver: Miller et al. (2011) 


Memphis: Olds et al. (2010) 
Elmira: Glazner et al. (2004) 
WSIPP (2018b) 


Parents as Teachers 
WSIPP (2018c) 


Follow-Up 
Period 


2 years 
7 years 
Lifetime projection 
Lifetime projection 


4 years 
4 years 
4.5 years 
9 years 


12 years 


15 years 
Lifetime projection 


Lifetime projection 


Benefit- 
Cost Ratio? 


-0.17 
0.15 
-4.20 
0.98 


0.55 
0.29 
0.26 
3.05 


1.07 


3.93 
1.63 


0.24 


Stakeholder 
Perspective® 


Society 
Government 
Society 
Society 


Government 
Government 
Government 
Society 


Government 


Government 
Society 


Society 


Main Source of Benefits (In Descending Order)° 


Not applicable 

Medicaid delivery and hospitalizations, public assistance 
Not applicable 

Maternal earnings associated with employment, K-12 
special education 


Not available 

Tax revenue, subsidized child care, Medicaid, food stamps 
Food stamps, foster care, AFDC 

Maternal earnings and employer-paid supplements, 
maternal depression 

Food stamps, Medicaid, AFDC/TANF 

Food stamps, AFDC, tax revenue, Medicaid 

Child earnings due to reduced child maltreatment, infant 
mortality 


Child earnings due to reduced child maltreatment 
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(continued) 


Table 3.3 (continued) 


SOURCE: Summaries and calculations based on results from the four evidence-based models included in 
MIHOPE. 


NOTES: WSIPP = Washington State Institute for Public Policy; AFDC = Aid to Families with Dependent 
Children; TANF = Temporary Assistance for Needy Families. 

This table includes results from benefit-cost analyses of the four evidence-based home visiting models 
included in MIHOPE. It includes original benefit-cost evaluations by model developers and independent 
evaluators and secondary evaluations by WSIPP. It does not include subgroup findings or analyses that 
include studies of models implemented outside the United States. 

The WSIPP results have been updated from those included in Michalopoulos, Faucetta, Warren, and 
Mitchell (2017) due to WSIPP's use of updated benefit-cost methods. 

@The benefit-cost ratios presented were calculated by the original study authors or WSIPP, with the ex- 
ceptions of Healthy Families Oregon, the Nurse-Family Partnership Elmira 4-year study, and the Nurse- 
Family Partnership Memphis 12-year study. In these cases MDRC calculated the ratios based on authors’ 
benefits and costs to avoid presenting results in different annual dollar amounts. 

>*Government” means the benefit-cost calculations considered only government expenditures and reve- 
nues. “Society” indicates a wider perspective, including outcomes such as those related to the earning of 
high school diplomas or equivalents. 

°Defined as the largest benefits that, taken together, comprise at least 75 percent of total benefits. 
Some of the information came from authors of the cited studies while other benefits were calculated by the 
authors of Michalopoulos, Faucetta, Warren, and Mitchell (2017). 


e Parents as Teachers: One WSIPP analysis. 


In Table 3.3, the main result is the benefit-cost ratio shown in the third column. 
This ratio is calculated by dividing the estimated monetary benefits by their costs. Ratios 
greater than 1.0 indicate that benefits exceed costs, and ratios less than 1.0 indicate 
that costs exceed benefits. Since this number is a ratio, it also indicates the degree to 
which benefits are greater or less than costs. For example, the benefit-cost ratio for the 
Healthy Families America study in New York, which is shown in the second row, is 0.15, 
indicating that benefits were only 15 percent as large as costs with a seven-year follow- 
up period. By comparison, the Denver study of Nurse-Family Partnership at 9 years has 
a benefit-cost ratio of 3.05, indicating that benefits were more than three times as great 
as costs. The second-to-last column shows whether the benefits are for government 
agencies (for example, reduced government costs) or society as a whole. Where the 
benefit-cost ratio is positive, the final column shows the main outcomes that contributed 
to estimated benefits. 
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Overall, these analyses have found: 


e The benefits of home visiting generally exceed its costs over pe- 
riods of 9 years or more. 


Home visiting programs incur costs right away, but participating parents and chil- 
dren can see improved outcomes over their lifetimes. Table 3.3 shows findings from 
Nurse-Family Partnership and Healthy Families America studies from several defined 
follow-up periods. All three Nurse-Family Partnership studies found that the benefits of 
home visiting were less than costs when children were 4 years of age but exceeded 
costs when children were 9 years of age or older. For Healthy Families America, the 
benefits with a 7-year follow-up are greater than with a 2-year follow-up (0.15 versus 
-0.17), although neither Healthy Families America estimate indicates that the monetary 
benefits exceeded the cost of providing services." 


« Benefits to the government generally stem from increased paren- 
tal earnings. 


Society benefits from improved parent and child well-being. For the longer-term 
follow-ups of Nurse-Family Partnership, for example, increased maternal earnings re- 
sulted in increased tax revenue and reduced spending on public assistance programs 
such as Medicaid, Temporary Assistance for Needy Families, and the Supplemental 
Nutrition Assistance Program. From society’s perspective, the 9-year follow-up in the 
Nurse-Family Partnership study in Denver found that benefits also came from reduced 
maternal depression, which can result in improvements in many aspects of parent and 
child well-being. '2 The WSIPP lifetime projections for Nurse-Family Partnership and Par- 
ents as Teachers indicate that society also benefited from reduced child maltreatment, 
which was projected to increase future earnings when these children become adults. 


e Projecting benefits over an individual’s lifetime introduces con- 
siderable uncertainty into these findings, especially with shorter 
follow-up periods. 


The lifetime projections are consequently quite variable, ranging from —4.20 
when Healthy Families America results in Oregon were projected from a 2-year follow- 
up, to 0.24 for Parents as Teachers, to 1.63 for Nurse-Family Partnership when project- 
ing from the longer follow-up periods available in those studies. Moreover, these pro- 


There are no Parents as Teachers studies that measured benefits using defined follow-up 
periods. 


'2Knitzer, Theberge, and Johnson (2008). 
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jected results can change. In an earlier MIHOPE-LT publication discussing these bene- 
fit-cost results, an earlier set of results from WSIPP indicated benefits exceeded costs 
over a lifetime for all three evidence-based models (Healthy Families America, Nurse- 
Family Partnership, and Parents as Teachers). *3 


Summary 


Previous studies of the four MIHOPE evidence-based home visiting models have pro- 
vided some information on the effectiveness of home visiting programs when children 
are in kindergarten and beyond. MIHOPE-LT can expand this body of evidence, as de- 
scribed in subsequent chapters. 


‘3The WSIPP benefit-cost ratios reported in Michalopoulos, Faucetta, Warren, and Mitchell (2017) 
are higher than those in this report. The benefit-cost ratios in the earlier MIHOPE-LT publication are 
1.25, 1.88, and 3.44 for Healthy Families America, Nurse-Family Partnership, and Parents as Teach- 
ers, respectively. These results are different because as mentioned above, WSIPP periodically 
changes its estimates based on updated benefit-cost methods and literature reviews. 
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Chapter 4 


Overview of Plans for MIHOPE Long-Term Follow-Up 


This chapter provides an overview of the plans for MIHOPE-LT. It presents the research 
questions that the study is designed to answer, the proposed timing of potential follow- 
up points, the sampling the study may use, and the data sources and outcome areas 
that may be examined. 


Research Questions 


The primary goal of MIHOPE-LT is to measure the long-term effects on family outcomes 
of having access to MIECHV-funded evidence-based home visiting through programs 
that participated in MIHOPE. The study also plans to determine whether effects differ for 
different subgroups of families and examine the pathways through which home visiting 
has long-term effects. In addition, a benefit-cost analysis is proposed. Discussion of the 
four primary research questions that MIHOPE-LT was designed to try to answer follows. 


1. What are the long-term effects of being assigned to receive evidence- 
based home visiting for families who enrolled in MIHOPE? 


MIHOPE-LT can contribute to the body of evidence on home visiting because of 
its large sample and because data will be collected the same way across the four evi- 
dence-based models (Early Head Start - Home-based option, Healthy Families Amer- 
ica, Nurse-Family Partnership, and Parents as Teachers). In addition, programs have 
evolved since the earlier studies were conducted, in part because of the statutory re- 
quirements of the MIECHV Program and in part due to ongoing quality improvement 
efforts by programs and models. The context in which programs operate and the partic- 
ipants in those programs have changed. Finally, the prior evidence of effectiveness is 
limited for any particular outcome, and MIHOPE-LT provides an opportunity to gain im- 
portant knowledge about these outcomes. 


2. Are the long-term effects of home visiting larger for some families than 
for others? 


Learning about effects for particular groups of families — that is, for subgroups 
by different types of family characteristics — could allow for better targeting of home 
visiting services. Although, as described in Chapter 2, analyses in MIHOPE using data 
collected when children were about 15 months of age suggest that the effects of home 
visiting do not vary much with family characteristics, MIHOPE-LT can continue to look 
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for subgroup differences. A review of the literature that informed the MIHOPE 15-month 
analysis found that the same subgroups have not been examined consistently in previ- 
ous studies of home visiting, so continuing to provide this information for the four evi- 
dence-based models could be informative. However, MIHOPE-LT may examine addi- 
tional subgroups that were not analyzed when children were 15 months of age. For 
example, information about mothers’ adverse experiences as children will be collected 
at the kindergarten follow-up and could be used to form additional subgroups. 


3. What are the pathways through which home visiting affects families’ 
longer-term outcomes? 


The longitudinal nature of MIHOPE-LT and its use of randomization provide a 
unique opportunity to investigate the pathways through which home visiting benefits 
families. 


Providing credible information on pathways may still be difficult because home 
visiting is intended to affect a wide range of outcomes, which are likely to interact in 
complicated ways over time. For example, maternal mental health could affect child well- 
being as well as other maternal behaviors (parenting, child maltreatment, intimate part- 
ner violence, education, employment, and earnings), all of which then also have the 
potential to affect child well-being. The same is true for many aspects of adult well-being. 
Improvements in one aspect of adult well-being can lead to improvements in other as- 
pects of adult well-being or can lead to improvements in child well-being. Child well- 
being then has the potential to continue to affect adult well-being. Home visiting could 
also have a direct effect on child well-being through, for example, early detection of de- 
velopmental delays. 


Although the statistical models that are used to analyze pathways might be com- 
plicated, they could further an understanding of how home visiting may affect policy- 
relevant outcomes. For example, at the kindergarten follow-up, the study could examine 
the pathways through which home visiting may affect school readiness by determining 
how positive impacts on families at the 15-month follow-up are related to school readi- 
ness as measured at the kindergarten follow-up. 


4. How do the monetary benefits of home visiting compare with its costs 
over the long term? 


Longer-term follow-up with the MIHOPE families could also allow the study to 
measure impacts that could be linked to long-term economic savings to the government 
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or economic benefits to families and society.’ Previous benefit-cost analyses of the mod- 
els that participated in MIHOPE have found that costs exceed benefits with less than 
nine years of follow-up data, but results from the Nurse-Family Partnership indicate that 
benefits exceed costs with longer follow-up.2 Thus, benefit-cost analyses in MIHOPE 
could be conducted at future points in time to reveal how the difference between benefits 
and costs changes as children get older. 


As in prior benefit-cost analyses of home visiting, the benefit-cost analysis could 
use three types of outcomes: 


e Outcomes readily expressed in monetary terms, such as earnings, 
public assistance (Temporary Assistance for Needy Families, Supple- 
mental Nutrition Assistance Program, and Medicaid), and health care 
use (for example, hospitalization and emergency department care) 


e Outcomes that have been assigned a monetary value, such as mater- 
nal depression (which can influence health care costs, employment, 
and public assistance), and child maltreatment (which can have costs 
to the child welfare system) 


e Outcomes that can be used to estimate future benefits, such as child 
maltreatment and special education to project improvements in chil- 
dren’s future earnings as adults and maternal alcohol use, and mater- 
nal graduation from high school to project improvements in maternal 
earnings 


Timing of Potential Follow-Up Points 


To determine the optimal timing of follow-up data collection to answer the key research 
questions and minimize the burden on families, the study team reviewed other long-term 
follow-up studies of home visiting programs, examined the existing literature on child 
and family outcomes of interest, and consulted with experts in home visiting research, 
early childhood and family intervention research, economics, and benefit-cost analysis. 


‘Cost data were collected during the implementation phase of MIHOPE, and the costs of deliver- 
ing home visiting services in the year after families began receiving services are discussed in Corso, 
Ingels, and Walcott (forthcoming). 

2Not counting the lifetime projections conducted by the Washington State Institute for Public Pol- 
icy, benefit-cost analyses have not been conducted for the other three evidence-based models with 
follow-ups that occurred when children were older than 7 years of age. 
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As a result of this work, four potential follow-up points were identified that are 
based on the MIHOPE child’s expected progression through school: kindergarten, third 
grade, middle school, and high school. The rationale for each of these time points is 
described below. 


In addition, brief check-ins with families (to obtain updated contact information 
and maintain engagement with the study) could occur periodically, and administrative 
data could be obtained throughout the follow-ups and could be collected past the last 
follow-up with families. 


Kindergarten: 


Measuring children’s cognitive, behavioral, self-regulatory, and social-emotional 
skills before formal schooling begins or at the outset of formal schooling will provide 
important data on the intermediate effects of home visiting. In addition, a wealth of liter- 
ature demonstrates that children’s math, language, and social-emotional skills at the 
transition to formal schooling are predictive of academic and behavioral outcomes in the 
longer term, and a follow-up in the children’s kindergarten year will allow the study team 
to measure these key mediators. Consistent with this evidence from the literature, the 
legislation that authorized MIECHV indicated that MIECHV-funded home visiting pro- 
grams are expected to improve school readiness. *® 


Third Grade 


Children’s behaviors and self-regulation in third grade are important predictors of 
future outcomes, such as academic achievement and aggressive behavior.® In addition, 
third grade generally presents the first opportunity to obtain school records that include 
reading and math state test scores, which are key predictors of future school success, 
as research has found empirical evidence for the association between reading skills at 
third grade and high school graduation rates.’ However, the previous studies of home 


3The kindergarten follow-up data collection began in January 2019 and will continue through 2022 
with families whose children are 5 years of age by the state cut-off date for entry into kindergarten 
each year. 

4Duncan et al. (2007); Eisenberg, Valiente, and Eggum (2010); Portilla et al. (2014). 

5SEC. 511 [42 U.S.C. 711] (c) (1) indicates that grants are to be made to enable eligible entities 
to deliver home visiting services in order to promote improvement in several outcome areas that in- 
clude school readiness. SEC. 511 [42 U.S.C. 711] (d) (1) (A) includes school readiness in the list of 
benchmark areas that eligible entities are expected to improve. 

6Kokko et al. (2006); Ladd and Dinella (2009); Miles and Stipek (2006); Valiente, Lemery-Chalfant, 
Swanson, and Reiser (2008). 

7Hernandez (2011). 
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visiting discussed in Chapter 3 have not found any effects on school performance after 
the age of 7 years. 


Obtaining information at or following third grade might be particularly important 
for answering the question of how the monetary benefits of home visiting compare with 
its costs. Because benefits may continue to accrue as children get older, obtaining in- 
formation about family well-being after the children’s kindergarten year might allow the 
study to measure more of the benefits. As discussed in Chapter 3, benefits can be meas- 
ured more precisely over longer follow-up periods, and studies of the Nurse-Family Part- 
nership have found that the benefits of home visiting services exceed their costs only 
when examined when children are nine years of age or older. Thus, a third-grade follow- 
up could provide information on whether that finding still holds with a larger sample and 
for additional models. Children’s test scores would be key measures for the benefit-cost 
analysis. Measures of family economic self-sufficiency, family violence, child maltreat- 
ment, child health, and maternal health would also contribute to this analysis. 


Middle School 


The transition to adolescence is considered an especially stressful time, charac- 
terized by challenging biological, cognitive, and psychosocial changes.® It is also a time 
when some children start engaging in risky health-related behaviors such as sexual ac- 
tivity, smoking, and drug use, and when early indicators of future juvenile justice involve- 
ment may emerge. 


In terms of prior evidence of effectiveness of home visiting, Nurse-Family Part- 
nership’s Memphis study is the only one for which there is evidence when children are 
12 years of age, with statistically significant impacts related to child development, child 
and maternal health, and economic self-sufficiency. 


High School 


Similar rationales point to the value of a follow-up when children are in high 
school. From the perspective of child development and academic achievement, the tran- 
sition to high school is another critical period,? which might suggest conducting a follow- 
up when the participating children are age 15. In addition to taking more advanced 
course work, children in high school must navigate a new and potentially larger social 


®Arnett (1999); Hall (1904); Schwartz, Cappella, and Seidman (2015). 
®Alspaugh (1998); Graber and Brooks-Gunn (1996); Smith (2006). 
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environment, a complex set of school structures, and a context that is likely less emo- 
tionally supportive than the settings they experienced in prior schooling. ‘° High school is 
also a time when teens might become pregnant or become parents, and involvement 
with the juvenile justice system may increase." 


Additionally, following up in high school could allow for the replication of evidence 
of reductions in child maltreatment and juvenile delinquency at age 15 in the Nurse- 
Family Partnership study in Elmira’? — the only follow-up with children this age in the 
seven studies included in Chapter 3. Finally, measuring outcomes for children at this 
point could provide evidence for the benefit-cost analysis on several outcomes that may 
result in reduced government spending and benefits to society (such as improved aca- 
demic achievement for children, improved child health and mental health, and reduc- 
tions in involvement with the criminal justice system). 


Sampling 


In making decisions about how to collect data about families, the study team has con- 
sidered the possibility of collecting some types of data for a subset of families (rather 
than collecting all data about all families) at each follow-up point.’ The primary reason 
to do this is to use resources where they would create the most value for the study. 


Using resources where they would create the most value is particularly important 
for MIHOPE-LT because the sample is so widely dispersed geographically (families en- 
rolled in MIHOPE at 87 locations in 12 states) and because the children in the sample 
range in age (with, for example, children in the MIHOPE sample entering kindergarten 
over the course of four years). 4 


As a result of these sample characteristics, the team has weighed the cost of 
collecting various types of data against the value those data are likely to provide to the 
study. Weighing the costs of data collection with the value they provide suggests using 
more resource-intensive methods with a subset of families. For example, assessments 
of children conducted in families’ homes could be done with a subset of families that is 


‘0Dupéré et al. (2015); Yeager et al. (2016). 

National Institute of Justice (2014). 

12Olds et al. (1997); Zielinski, Eckenrode, and Olds (2009). 

'8This section describes sampling strategies that have been considered by the team. The sam- 
pling strategies used in the kindergarten follow-up are described in Chapter 5. 


4An eighty-eighth site participated in the study but did not enroll any families. Both the length of 
the study’s enrollment period and the range of child and gestational ages at enrollment contribute to 
this four-year period. 
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still large enough to estimate the effects of the programs overall but not to investigate 
subgroup differences. In contrast, when costs do not vary much with the number of fam- 
ilies for whom data are collected (such as with an administrative data source like the 
National Directory of New Hires), the study could collect data for all families. 


Subsampling may take several forms: 


e The study could focus on obtaining school records only in states with 
centralized data systems. 


e For data collection that requires going to a family’s home (as would be 
done with direct assessments of children or observed mother-child in- 
teractions), the study could focus on locations with enough families to 
be able to efficiently use field staff. 


The subsampling described above would not be random since it would favor lo- 
cations where data collection is less expensive compared with locations where costs 
would be greater. However, it would not affect the random assignment design of the 
study since families were randomly assigned to program and control groups within each 
local home visiting program. In addition, the study team is considering reducing the over- 
all burden of data collection by randomly choosing which families will be asked to provide 
which type of information. The basic idea behind this strategy — which is sometimes 
referred to as “planned missingness” — is that collecting some information from a subset 
of the sample allows the study to collect a wider set of information than it could other- 
wise, while state-of-the-art methods for working with missing data minimize the extent 
to which the precision of the impact estimates is affected.'® 


One example of planned missingness is the multiform design, in which multiple 
forms of a survey are each conducted with a randomly chosen subset of families. ’* In 
MIHOPE, the multiform design could be applied by asking each family a subset of ques- 
tions from a scale. For example, considering a scale with nine items, one-third of sample 
members could be asked to answer items one, two, four, and seven; one-third could be 
asked to answer items one, two, five, and eight; and one-third could be asked to answer 
items one, two, three, six, and nine. Asking a subset of questions on the scale would still 


‘SWith planned missingness, statistical methods can be used as a way to fill in the missing infor- 
mation. To the extent that other information such as baseline data or other outcome data can help 
predict the outcome, these statistical methods can reduce the loss of statistical power that results 
when the outcome data for only part of the sample are collected. Strong predictors would result in little 
loss of power while very weak predictors would produce mostly random information on missing out- 
comes, and in that case planned missingness would result in a substantial reduction in statistical pre- 
cision. See Lang and Little (2016). 


*6Graham, Taylor, Olchowski, and Cumsille (2006). 
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provide an unbiased estimate of the underlying construct but leave more room on the 
survey to measure other constructs. 


Data Sources 


Home visiting has the potential to affect a wide range of outcomes. In determining the 
scope of the MIHOPE-LT data collection efforts, the team considered a wide range of 
data sources. These include data sources that require direct contact with families, such 
as structured interviews with mothers, ‘’ direct assessments of children and mothers, in- 
home observations, video-recorded mother-child interactions, qualitative interviews, bi- 
ospecimens (such as hair or saliva specimens), and, in potential later follow-up waves, 
structured interviews with children. Teacher surveys, which would require direct contact 
with individuals who were not previously part of the study, were also considered, as were 
a variety of administrative data sources: Medicaid records, state child welfare records, 
data from the National Directory of New Hires (to measure employment and earnings), 
juvenile justice records, school records, tax records, mortality records, and the American 
Community Survey. 


The sources of data to be collected at the MIHOPE-LT kindergarten follow-up 
are shown in Chapter 5. If follow-ups are conducted at later time points, the data sources 
will be chosen after key outcomes are finalized and resources are allocated. 


As indicated earlier, administrative data could be collected throughout children’s 
childhood and adolescence and past high school. Once the MIHOPE children who are 
the focus of this study are 18 years of age, consent could be obtained from them for 
collecting administrative data during their adulthood. 


Summary 


Chapter 4 provides an overview of the design for long-term follow-up with MIHOPE fam- 
ilies. The chapter describes the four research questions MIHOPE-LT is designed to an- 
swer and the rationale for following up with families when children are in kindergarten, 
as well as at three later potential time points. The next chapter describes the design for 
the kindergarten follow-up point in detail. Sampling considerations are also discussed. 


‘7In cases where the mother is not available (for example, because she no longer has custody of 
the child), data collection would be conducted with the child’s primary caregiver, such as the child’s 
father. This approach was also used at the MIHOPE follow-up that occurred when children were 15 
months of age. 
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Chapter 5 
The MIHOPE-LT Kindergarten Follow-Up 


This chapter covers the data collection plans for the kindergarten follow-up point. It be- 
gins by describing the considerations that informed the constructs included in the kin- 
dergarten data collection effort and then describes what will be examined in each out- 
come area. It then details the data sources that will be used to obtain findings about 
these constructs. The chapter concludes with a discussion of the sample distribution 
across multiple cohorts, the sampling plan, the specific timing of the data collection effort 
within the kindergarten year, and the analyses that the study team plans to conduct with 
the kindergarten data. 


Choosing Constructs to Measure in Kindergarten 


Home visiting has the potential to affect a wide range of outcomes. The study team 
generated a lengthy list of potential constructs by reviewing previous studies of home 
visiting, compiling the logic models for the four MIHOPE evidence-based models (Early 
Head Start - Home-based option, Healthy Families America, Nurse-Family Partnership, 
and Parents as Teachers), discussing long-term effects with the model developers and 
other key stakeholders, and reviewing the broader literature in each domain of interest. 
It was not possible to examine all the constructs at the kindergarten follow-up,’ so the 
study team considered prior evidence of effects, potential use in a benefit-cost analysis, 
adverse childhood experiences (ACEs), and mediators and moderators, as explained 
below, when deciding which constructs to examine. 


Previous Evidence of Effects 


As noted earlier, one of the goals for MIHOPE-LT is to learn about the long-term 
effects of being given access to evidence-based home visiting. Consequently, prior evi- 
dence of effectiveness was considered when choosing among constructs, and the focus 
in reviewing the evidence from previous studies was on examinations that occurred 
around kindergarten age, because findings from the same time period in children’s lives 
were considered the most relevant for the MIHOPE kindergarten design. The results 


‘Constructs that were considered but are not included are discussed in Appendix D. 

Policy relevance was considered but was not used to narrow the kindergarten outcomes list be- 
cause all the outcomes considered are policy-relevant. The ACEs criterion enables the study to cap- 
ture many outcomes that are most consequential for children’s well-being, and the benefit-cost lens 
enables the study to capture effects on government assistance programs. 
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from the MIHOPE 15-month impact analysis were also considered, but because the 15- 
month findings were not available until after many decisions about the kindergarten de- 
sign had been made and were similar to effects in previous studies, and because it is 
difficult to anticipate how effects at 15 months may affect families when children are in 
kindergarten, the evidence from previous studies was given more weight.’ 


Potential Use in Benefit-Cost Analysis 


Because conducting a benefit-cost analysis is one of the priorities of MIHOPE- 
LT, the study team also considered the potential for a construct to be linked to long-term 
economic benefits. In prior benefit-cost analyses, home visiting may not have had a sta- 
tistically significant impact on constructs that have been used either to directly measure 
benefits or to model future benefits. Because a benefit-cost analysis can use constructs 
that may not have statistically significant impacts, the study team included a construct’s 
potential use in benefit-cost analyses as a separate criterion. 


Adverse Childhood Experiences 


Given research that shows that the risk for a host of poor health and well-being 
outcomes increases as the number of adverse childhood experiences (ACEs) in- 
creases, a construct’s ability to contribute to an ACE index was also considered.* The 
range of constructs considered to be ACEs has evolved since the original ACE study, 
but they often include topics such as abuse (emotional, physical, sexual), having a 
mother who has been treated violently, substance use in the household, mental illness 
in the household, parental separation or divorce, incarceration of a household member, 
emotional neglect, and physical neglect. By including measures of children’s ACEs at 
the kindergarten follow-up, the study team will be able to measure whether home visiting 
affects these predictors of subsequent negative outcomes (outcomes that could emerge 
as the MIHOPE children become adults).§ 


3Similarly, because it is difficult to anticipate how effects in earlier waves of prior studies affected 
families when children were in kindergarten, findings from follow-ups that occurred at younger ages in 
prior studies were not considered. 

4Felitti et al. (1998). 

5Separately, the study team is also obtaining information about mothers’ ACEs at the kindergarten 
follow-up, to learn about their childhood experiences. 
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Mediators and Moderators 


As indicated in Chapter 4, a goal of MIHOPE-LT is to investigate the pathways 
through which home visiting has long-term effects on families. To examine these path- 
ways, the study team needs to measure mediators — mechanisms that can explain the 
relationship between home visiting and long-term effects. Because home visiting is an 
intervention that attempts to influence a wide range of behaviors, and it is likely that 
outcomes will be influenced through multiple mediators, all the constructs considered for 
the kindergarten follow-up have the potential to be mediators of the longer-term effects 
of home visiting. However, to ensure that important mediators of policy-relevant out- 
comes in third grade, middle school, and high school are being measured at kindergar- 
ten, a few constructs were included at the kindergarten follow-up, primarily because of 
their value as mediators of the effects of home visiting at later follow-up points. In addi- 
tion, a few constructs that have the potential to moderate — or explain the strength of 
the relationship between home visiting and long-term effects — were included at the 
kindergarten follow-up. 


Kindergarten Constructs 


As previously indicated, home visiting programs attempt to support families in a variety 
of ways, tailoring services to meet families’ needs. Although individual services may 
vary, the study team identified eight areas of adult and child functioning and behavior 
where effects of home visiting services are most likely to be observed when children are 
kindergarten age:® 


®These eight areas cover aspects of the outcome areas specified in the MIECHV legislation, even 
though they do not align exactly with those areas. The study team modified the outcome areas for the 
kindergarten follow-up so that the areas would reflect the more narrow focus of the constructs of in- 
terest at the kindergarten follow-up. (For example, “maternal behavioral health” is used at kindergar- 
ten, rather than the broader “prenatal, maternal, and newborn health.”) A list of the kindergarten out- 
come areas follows, with the legislative outcome areas with which they are most closely aligned shown 
in parentheses: family economic self-sufficiency (family economic self-sufficiency); maternal positive 
adjustment (prenatal, maternal, and newborn health; parenting skills); maternal behavioral health (pre- 
natal, maternal, and newborn health); family environment and relationship between parents (family 
economic self-sufficiency; crime and domestic violence); parent-child relationship and interactions 
(parenting skills; child health and development); parental support for cognitive development (parenting 
skills); child functioning (child health and development; school readiness); and receipt of and connec- 
tion to services (child health and development; school readiness). 


47 


e Family economic self-sufficiency 

e Maternal positive adjustment’ 

e Maternal behavioral health® 

e Family environment and relationship between parents 
e Parent-child relationship and interactions® 

e Parental support for child’s cognitive development 

e Child functioning (school readiness) 

e Receipt of and connection to services 


Constructs in two additional areas are also being measured at the kindergarten 
follow-up: 


e Social support 
e School and neighborhood context 


These last two areas were included primarily because they can provide important 
context and information about characteristics that may moderate the long-term impacts 
of home visiting. 


Information on the measurement of all the kindergarten constructs, including dis- 
cussion of scales and assessments, is included in Appendix A. 


Table 5.1 shows all the constructs that will be examined at the kindergarten fol- 
low-up and their data sources. 1° 


7The term “maternal positive adjustment” is used here to refer to aspects of maternal functioning 
such as parenting stress, mastery, self-regulation, and household chaos. 


’This area includes maternal mental health and maternal substance use and alcohol use. 
°Severely negative parenting behavior (including child maltreatment) is discussed in this area. 


10AIl these constructs are being measured, but decisions about outcomes that will be analyzed 
have not yet been made. The team intends to produce an analysis plan that will include this information 
and will consider effects found in prior MIHOPE waves when making decisions about outcomes. 
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Table 5.1 
Constructs Measured at the MIHOPE-LT Kindergarten Follow-Up 


Outcome Area and Construct 


Data Source 


Family economic self-sufficiency 
Public assistance receipt@ 


Employment and earnings 

Income 

Material hardship 

Food insecurity 

Housing status and mobility 
Highest level of education 
Subsequent pregnancies and births 


Maternal positive adjustment 
Mastery 


Mobilizing resources 

Parenting stress 

Parent-child separations 

Household chaos 

Self-regulation (working memory) 
Child school attendance and tardiness 


Maternal behavioral health 
Depressive symptoms 

Drug use 

Alcohol use 


Family environment and relationship between parents 
Mother's relationship status 


Mother's relationship with biological father of child 
Family conflict 

Physical violence: perpetration 

Physical violence: victimization 

Experience with battering 


Parent-child relationship and interactions 


Structured interview, CMS MAX files® 
Structured interview, NDNH 


Structured interview 
Structured interview 
Structured interview 
Structured interview 
Structured interview 
Structured interview 


Structured interview 
Structured interview 
Structured interview 


Structured interview, state child welfare records 


Structured interview 


Direct mother assessment 
Teacher survey, school records 


Structured interview 
Structured interview 
Structured interview 


Structured interview 
Structured interview 
Structured interview 
Structured interview 
Structured interview 
Structured interview 


Parental warmth 
Parent-child interaction® 
Abuse (physical, sexual) 
Psychological aggression 
Neglect 


Structured interview, observer ratings 
Video-recorded mother-child interaction 
Structured interview, state child welfare records? 
Structured interview, state child welfare records 
State child welfare records 


Parental support for child's cognitive development 
Home literacy environment 


Cognitive stimulation 


Structured interview 
Structured interview, video-recorded mother- 
child interaction 


(continued) 
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Table 5.1 (continued) 


Outcome Area and Construct 


Data Source 


Child functioning (school readiness 
Behavior problems 

Social-emotional skills 

Learning behaviors and approaches to learning 
Disciplinary incidents 

Executive function 

Math skills 

Language skills 


Receipt of and connection to services 
Child received any early intervention services 


Child care setting before kindergarten 

Child has health insurance coverage 

Child emergency department visits 

Child hospitalizations 

Child receiving any special education services/has an IEP 


Social support 
Involvement of the biological father or father figure with the child 


Social support 


School and neighborhood context 
School characteristics 


Neighborhood disadvantage 


Structured interview, teacher survey 
Structured interview, teacher survey 
Teacher survey 

Teacher survey, school records 

Direct child assessment, observer ratings 
Direct child assessment 

Direct child assessment 


Structured interview 

Structured interview 

Structured interview, CMS MAX files 
Structured interview, CMS MAX files 
Structured interview, CMS MAX files 
Teacher survey, school records 


Structured interview 
Structured interview 


Common Core of Data or Office of Civil Rights data 
American Community Survey 


NOTES: CMS MAX files = Centers for Medicare & Medicaid Services Medicaid Analytic eXtract files; NDNH = National 


Directory of New Hires; IEP = individualized education program. 


Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. 
*The benefits that will be measured are those from the Supplemental Nutrition Assistance Program; disability insur- 
ance; Temporary Assistance for Needy Families; Special Supplemental Nutrition Program for Women, Infants, and Chil- 


dren; and Medicaid. 


>Medicaid receipt is being measured using the CMS MAX files. Receipt of other types of public assistance is only be- 


ing measured on the structured interview. 


‘Includes measures of parental sensitivity, parental intrusiveness, parental negative regard, parental positive regard, 
parental detachment, child engagement of parent, child's quality of play, and child negativity toward parent. 
4|nformation on sexual abuse will not be collected from the structured interview. 


The sections that follow describe the constructs that will be measured in each 
outcome area and the rationale for their inclusion. The tables display information about 
three of the reasons that constructs are being measured at the kindergarten follow-up: 
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(1) prior evidence of effects,’ (2) potential use in the benefit-cost analysis,'? and (3) 
measures of ACEs. Constructs’ value as mediators or moderators of the effects of home 
visiting are discussed in the text. 


Family Economic Self-Sufficiency 


Home visiting programs aim to increase families’ economic security by connect- 
ing them to services or benefits when needed, by helping them reduce their dependence 
on public benefits, by connecting them to employment opportunities, or by supporting 
mothers in their education and training endeavors — all with the goal of eventually im- 
proving employment opportunities and income. The type of support that home visitors 
provide depends on families’ individual needs, and it is possible that effects on con- 
structs such as maternal educational attainment will take some time to emerge because 
completion of a degree or program could take years. Several constructs in the area of 
family economic self-sufficiency will be measured. As shown in Table 5.2, many of the 
constructs in this area could contribute to the benefit-cost analysis, in part because they 
are naturally expressed in monetary terms or can be easily translated into monetary 
terms. For example, there is literature on the adverse consequences of short intervals 
between births, including worsened economic and employment outcomes. ‘* Measures 
of material hardship — including the ability to pay rent or pay for utilities, pay for health 
care, or afford sufficient food (“food insecurity”) — are indicators of a family’s degree of 
economic security and of a family’s connection to services and benefits that are direct 
targets of home visiting programs. The positive effects on food insecurity at the MIHOPE 
follow-up that occurred when children were about 15 months of age suggest that effects 
on this or other measures of material hardship may be found at the kindergarten follow- 
up. 1 


"Evidence from previous studies of the four MIHOPE evidence-based models from follow-ups 
with families who had children between 5 and 6 years of age is included in the table. The number of 
statistically significant effects and the total number of examinations are shown. 

12A final determination about the constructs that will be included in the benefit-cost analysis will 
be made before the analysis is conducted. 

'3Increased birth spacing will be examined, and, for children who were born after the child asso- 
ciated with the MIHOPE study, information on birth weight and gestational age at birth and whether 
they were admitted to the neonatal intensive care unit will be collected, given that these measures 
have implications for the benefit-cost analysis. 

“4Conde-Agudelo, Rosas-Bermtdez, and Kafury-Goeta (2007); Khoshnood et al. (1998); Polit 
and Khan (1986). 

‘SEood insecurity was the only measure of material hardship included at the MIHOPE 15-month 
follow-up. 
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Table 5.2 


Family Economic Self-Sufficiency Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects@ Benefit-Cost Analysis ACEs 
Public assistance receipt® 2 of 4 Yes No 
Employment and earnings 0 of 3 Yes No 
Income 1 of 1 Yes No 
Material hardship Not examined No No 
Food insecurity Not examined No No 
Housing status and mobility 0 of 1 No No 
Highest level of education 0 of 1 Yes No 
Subsequent pregnancies and births 3 of 7 Yes No 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013); Drazen and 
Haust (1993); Jones Harden, Chazan-Cohen, Raikes, and Vogel (2012); and Olds, Kitzman et al. (2004). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

®Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 

>The benefits that will be measured are those from the Supplemental Nutrition Assistance Program; disability 
insurance; Temporary Assistance for Needy Families; Special Supplemental Nutrition Program for Women, In- 
fants, and Children; and Medicaid. 


Maternal Positive Adjustment 


Home visiting programs aim to improve maternal functioning by increasing mothers’ 
sense of mastery,’ their ability to self-regulate,'’ and their ability to use resources in 
their community.’ (See Table 5.3.) These programs also try to reduce parents’ levels of 
stress by providing them with support and by helping them to learn and improve their 
parenting skills and knowledge about age-appropriate norms and expectations for their 
children. Improvements in maternal functioning may also lead to more ordered house- 
holds and more on-time attendance at school for children. Though maternal positive 
adjustment encompasses areas that are direct targets of home visiting, many of these 
constructs have not often been examined in previous studies of home visiting and were 
not examined at the MIHOPE follow-up that occurred when children were 15 months of 


‘6Mastery measures the extent to which a person thinks life chances are under her control. 


'7Self-regulation is defined as emotion and cognitive control capacities. See Blair and Diamond 
(2008); Blair and Raver (2015). 


‘8This ability includes feeling empowered to gain access to resources and being aware of their 
availability. 
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Table 5.3 


Maternal Positive Adjustment Constructs 


Potential Use in the Measure of 

Construct Evidence of Effects® Benefit-Cost Analysis ACEs 
Mastery 0 of 1 No No 
Mobilizing resources Not examined No No 
Parenting stress Not examined No No 
Parent-child separations Not examined Yes Yes 
Household chaos Not examined No No 
Self-regulation (working memory) Not examined No No 

Child school attendance and tardiness 1 of 1 No No 


SOURCES: Evidence of effects based on results from Drazen and Haust (1993) and Olds, Kitzman et al. (2004). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

4Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 


age. MIHOPE-LT will fill this gap, as these constructs have important implications as 
mediators of child outcomes. For example, parenting stress is a risk factor for child mal- 
treatment, '? and the degree of household order is associated with kindergarteners’ early 
reading skills,2° general cognitive functioning,?' early executive functioning skills,22 and 
the social-emotional development of preschool and school-age children and young ad- 
olescents.23 


In addition, participation in home visiting programs could affect mothers’ ability 
to retain custody of their children by reducing mothers’ criminal behavior or improving 
their ability to care for their children in a way that is not neglectful or abusive. Two of 
these measures of maternal positive adjustment — the mother’s incarceration or the 
child’s placement in foster care — have been included in the kindergarten follow-up and 
can provide relevant information for the benefit-cost analysis. 


‘9Rodriguez and Richardson (2007); Taylor, Guterman, Lee, and Rathouz (2009). 
20 Johnson, Martin, Brooks-Gunn, and Petrill (2008). 

tHart, Petrill, Deater-Deckard, and Thompson (2007). 

22\/ernon-Feagans, Willoughby, and Garrett-Peters (2016). 


?3See Coldwell, Pike, and Dunn (2006) for findings on preschool- and school-age children; see 
Dumas et al. (2005) for findings on young adolescents. 
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Maternal Behavioral Health 


Home visiting programs try to support mothers who may have behavioral health 
difficulties by screening and referring them to other services as needed. At the time of 
the kindergarten follow-up, impacts can potentially be observed on both maternal de- 
pressive symptoms and maternal substance use (including drugs and alcohol), as early 
impacts seen at the follow-up that occurred when children were 15 months of age could 
continue to be manifested at this point. Constructs in this area are also ACEs and can 
be used in the benefit-cost analysis. (See Table 5.4.) 


Table 5.4 


Maternal Behavioral Health Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects? Benefit-Cost Analysis ACEs 
Depressive symptoms 0 of 25 Yes Yes 
Drug use 0 of 3° Yes Yes 
Alcohol use 0 of 3° Yes Yes 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013) and Olds, Kitz- 
man et al. (2004). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

4Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 

>The previous outcome "mental health" is also included in this count. 

°Two previous studies examined effects on a combination of drug and alcohol use, so these outcomes are 
counted in the evidence of effects for both constructs. 


Family Environment and Relationship Between Parents 


Home visitors may work with mothers to support and encourage healthy relationships 
with the biological father of their child or with their current partner. (See Table 5.5.) Ef- 
fects of home visiting may therefore be seen on measures of family stability. (Lack of 
family stability is also an ACE and was identified as a key moderator of the effects of 
home visiting.)24 Home visitors can also work on reducing conflict and violence between 
partners if this stressor is present in families’ lives. Measurement of the physical acts of 


*4Parental separation or divorce is often included in measures of ACEs. 
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Table 5.5 


Family Environment and Relationship Between Parents Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects® Benefit-Cost Analysis ACEs 
Mother's relationship status 0 of 2 No Yes 
Mother's relationship with biological 
father of child 0 of 1 No Yes 
Family conflict Not examined No No 
Physical violence: perpetration Not examined No Yes 
Physical violence: victimization 0 of 2° No Yes 
Experience with battering Not examined No No 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013) and Olds, Kitz- 
man et al. (2004). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

®Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 

>Previous outcomes included under this construct are whether the child witnessed any violence and whether the 
mother was involved in any domestic violence. 


violence (which are ACEs) will be complemented by measurement of battering and fam- 
ily conflict.2° (Family conflict was identified as a key mediator of the effects of home vis- 
iting on child maltreatment.) 


Parent-Child Relationship and Interactions 


Constructs related to the parent-child relationship and parent-child interactions 
have rarely been examined in previous home visiting studies that occurred at age 5 or 
later, though these parent behaviors are direct targets of home visiting. The MIHOPE- 
LT kindergarten follow-up is trying to fill this gap by examining whether there are effects 
of home visiting on parenting several years after program enrollment and whether these 
effects mediate the longer-term effects of home visiting. Both positive and negative 
parent-child interactions can be measured; the constructs that measure maltreatment of 
children will be used in the benefit-cost analysis and are also ACEs. (See Table 5.6.) 


*5Battering is a “syndrome of control and increasing entrapment that accompanies the use of 
physical force in intimate relationships” (Smith, Earp, and DeVellis, 1995; Stark and Flitcraft, 1991), 
measured here using items from the Women’s Experience with Battering scale (Smith, Earp, and 
DeVellis, 1995). Family conflict is the amount of openly expressed anger and lack of cohesion among 
family members, measured here using items from the Family Environment Scale (Moos and Moos, 
2009). 
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Table 5.6 


Parent-Child Relationship and Interactions Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects? Benefit-Cost Analysis ACEs 
Parental warmth Not examined No No 
Parent-child interaction? 0 of 4 No No 
Abuse (physical, sexual) 0 of 3° Yes Yes 
Psychological aggression Not examined Yes Yes 
Neglect 0 of 3° Yes Yes 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013); Drazen and 
Haust (1993); and Jones Harden, Chazan-Cohen, Raikes, and Vogel (2012). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

4Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 

Includes measures of parental sensitivity, parental intrusiveness, parental negative regard, parental positive 
regard, parental detachment, child engagement of parent, child's quality of play, and child negativity toward parent. 

°One previous study examined three effects on a combination of abuse and neglect, so these outcomes are 
counted in the evidence of effects for both constructs. 


Parental Support for Child’s Cognitive Development 


All four evidence-based models in MIHOPE have the goal of supporting parents’ 
knowledge about early childhood development and developmentally appropriate parent- 
ing practices. The MIHOPE-LT kindergarten follow-up will examine whether home visit- 
ing improves parenting practices supportive of early learning, including environmental 
factors that are most influential for literacy growth and engaging in activities that have 
been shown to have positive effects on children’s later cognitive development and aca- 
demic achievement.26 (See Table 5.7.) 


Child Functioning (School Readiness) 


By supporting mothers’ well-being and positive parenting practices, home visiting 
might improve child outcomes as well, and improved child functioning might persist as 


6For findings on children’s cognitive development, see Fiorini and Keane (2014) and Kalil and 
Mayer (2016). For findings on academic achievement as children age, see Bodovski and Farkas 
(2008). 
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Table 5.7 


Parental Support for Cognitive Development Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects? Benefit-Cost Analysis ACEs 
Home literacy environment 3 of 3 No No 
Cognitive stimulation 1or1 No No 


SOURCE: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

8Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 


children make the transition into kindergarten. MIHOPE-LT is measuring children’s 
school readiness at kindergarten in a variety of domains such as behavior (problem be- 
haviors and social-emotional skills), language and literacy skills, math skills, executive 
functioning skills, and approaches to learning. (See Table 5.8.) School-readiness do- 
mains were selected largely based on evidence from past long-term home visiting stud- 
ies at the kindergarten time point, as well as hypotheses generated from current devel- 
opmental literature.2” All the constructs in this area are potential mediators of later 
academic achievement.?8 


Receipt of and Connection to Services 


Three of the constructs in this area will be used to inform the benefit-cost analy- 
sis, but they also have the potential to function as mediators of longer-term outcomes. 
Home visiting programs can directly connect families to services; they also have the 
potential to affect a mother’s ability to connect her child to services that could mediate 
later development, such as special education, attendance at a formal child care setting 
before entering kindergarten, and health insurance coverage. (See Table 5.9.) 


Social Support 


Home visiting has the potential to affect perceived social support (Such as the 
mother’s perception of social support networks and connections to communities and 


27Other domains, such as motor skills, had less support for their inclusion. 
?8Duncan et al. (2007); Eisenberg, Valiente, and Eggum (2010); Portilla et al. (2014). 
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Table 5.8 


Child Functioning (School Readiness) Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects@ Benefit-Cost Analysis ACEs 
Behavior problems 2 of 11 No No 
Social-emotional skills 0 of 3 No No 
Learning behaviors and approaches to learning 1 of 2 No No 
Disciplinary incidents Not examined No No 
Executive function 2 0f 5 No No 
Math skills 0 of 4 No No 
Language skills 4 of 11 No No 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2013); Drazen and 
Haust (1993); Jones Harden, Chazan-Cohen, Raikes, and Vogel (2012); Olds, Kitzman et al. (2004); Olds, 
Holmberg et al. (2014); and Sidora-Arcoleo et al. (2010). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

*Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 


Table 5.9 


Receipt of and Connection to Services Constructs 


Potential Use in the Measure of 

Construct Evidence of Effects@ Benefit-Cost Analysis ACEs 

Child received any early intervention services 0 of 1° No No 

Child care setting before kindergarten 1 of 3° No No 

Child has health insurance coverage Not examined No No 

Child emergency department visits Not examined Yes No 

Child hospitalizations Not examined Yes No 

Child receiving any special education 

services/has an IEP 0 of 4 Yes No 


SOURCES: Evidence of effects based on results from Chazan-Cohen, Raikes, and Vogel (2012); Drazen and 
Haust (1993); Jones Harden, Chazan-Cohen, Raikes, and Vogel (2012); and Olds, Kitzman et al. (2004). Four- 
year follow-ups are also included for the early intervention and child care setting constructs (Olds, Robinson et al., 
2004) because they are comparable to the time period that MIHOPE-LT will examine for these constructs. 


NOTES: ACE = Adverse childhood experience. IEP = individualized education program. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

4Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 

>A previous study examined effects on a combination of early intervention services and child care setting, so 
these outcomes are counted in the evidence of effects for both constructs. Counts from studies that looked at 
these constructs are not included in Table 3.2 because they are not about children who are 5 years of age or 
older. 
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community agencies) and the involvement of the biological father or father figure with 
the MIHOPE child. However, these constructs are primarily being examined because 
they will provide important context about family functioning and they have the potential 
to moderate the effects of home visiting. (See Table 5.10.) 


Table 5.10 


Social Support Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects@ Benefit-Cost Analysis ACEs 
Involvement of the biological father or father 
figure with the child Not examined No No 
Social support Not examined No No 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

'Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 


School and Neighborhood Context 


Home visiting is not expected to affect constructs related to school and neighborhood 
characteristics (shown in Table 5.11), but these constructs will provide important con- 
textual information. For example, some of the data that are available in school records, 
such as number of disciplinary incidents, are affected by district and school policies and 
should therefore be examined in the context of information about children’s schools and 
school districts.2? Information on the socioeconomic characteristics of families’ neighbor- 
hoods can be gathered from data sources such as the American Community Survey.*° 


°This information can be obtained from the Common Core of Data 
(https://nces.ed.gov/ccd/aboutccd.asp) or the Office of Civil Rights (httos://ocrdata.ed.gov). 

3°For example, the American Community Survey can be used to create a measure of socioeco- 
nomic disadvantage using the following four variables: percentage over age 25 without a high school 
degree, percentage unemployed, percentage of families living in poverty, and percentage receiving 


public assistance (Turney and Harknett, 2010). See https:/Awww.census.gov/programs-surveys/acs 
for more information on the American Community Survey. 
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Table 5.11 


School and Neighborhood Context Constructs 


Potential Use in the Measure of 
Construct Evidence of Effects? Benefit-Cost Analysis ACEs 
School characteristics Not examined No No 
Neighborhood disadvantage 0 of 2 No No 


SOURCE: Evidence of effects based on results from Hanks et al. (2011). 


NOTES: ACE = Adverse childhood experience. 

Information about how each construct is measured in MIHOPE-LT is available in Appendix A and B. More infor- 
mation on the outcomes counted under "evidence of effects" is available in Appendix E. 

8Defined as the number of favorable, statistically significant examinations out of the total number of examina- 
tions for each construct. Statistically significant results are those with a p-value of less than 0.05. 


Data Sources 


The data sources for the kindergarten follow-up are: 


a structured interview with the child’s mother to measure a broad set of 
constructs that are mostly not available from other data sources*" 


direct assessments of children’s language skills, math skills, and exec- 
utive function by trained interviewers 


interviewer observations of parental warmth and children’s self- 
regulation (for example, attention and impulse control) 


video-recorded mother-child interaction 


a direct assessment of mothers’ working memory by trained interview- 
ers 


a teacher survey 


federal administrative data on Medicaid22 


3'lIn cases where the biological mother no longer has custody of the child, data collection will be 
conducted with the child’s primary caregiver, such as the child’s father. This approach was also used 
at the MIHOPE follow-up when children were 15 months of age. 

°2The study team is planning to obtain Medicaid and Children’s Health Insurance Program (CHIP) 
data from the Centers for Medicare and Medicaid Services (CMS) Medicaid Analytic eXtract (MAX) 
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e federal administrative data on employment covered by the unemploy- 
ment insurance system (National Directory of New Hires) 


e state child welfare records 
e school records 


Tables in Appendix B show the constructs that will be obtained from each data 
source, and Appendix C contains more information about each of the administrative data 
sources. 


Kindergarten Cohorts 


As indicated in Chapter 4, MIHOPE includes children who enter kindergarten over the 
course of four years because of the length of the study’s enrollment period and the range 
of child and gestational ages at enrollment. Approximately 600 children entered kinder- 
garten in fall 2018 (cohort 1), about 2,100 children in fall 2019 (cohort 2), and about 
1,200 children in fall 2020 (cohort 3), and about 150 children will enter kindergarten in 
fall 2021 (cohort 4). 


Sampling 


Beginning with the first kindergarten cohort, a multiform design will be used to increase 
the amount of information that the structured interview can capture. No additional sam- 
pling is planned for the first cohort, but other forms of planned missingness may be used 
in subsequent cohorts. 


Timing 
A fall data collection period was chosen because it is a priority to measure the MIHOPE 


sample’s readiness for kindergarten and to measure program effects before the transi- 
tion to kindergarten has occurred. 


Due to logistical issues, data collection began with the approximately 600 fami- 
lies who are in the first of four cohorts of families in January 2019. Data is planned to 
begin in the fall for subsequent cohorts. Though the timing of the data collection periods 


files. In December 2019, these data changed to the CMS MAX files and Transformed Medicaid Sta- 
tistical Information System Analytic file (TAF) data but are referred to as the CMS MAX files throughout 
this report. 
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will not line up exactly across cohorts, the same constructs will be measured across all 
cohorts. 8 


Administrative data from the full kindergarten school year will be obtained. 


Planned Analyses 


Following the kindergarten data collection, the study team plans to conduct an impact 
analysis and mediational analyses (that is, analyses of the pathways through which an 
intervention produces effects).** Some principles for conducting the analyses are de- 
scribed below, but as in earlier waves of the study, the team intends to produce an anal- 
ysis plan.% 


Impact Analysis 


The impact analysis will assess the effectiveness of early childhood home visiting 
programs in improving the outcomes of families and children when children are in kin- 
dergarten, both overall and across key subgroups of families and programs. Random 
assignment was used in MIHOPE to create program and control groups that were ex- 
pected to be similar in all respects when they entered the study. As is standard in ran- 
dom assignment studies, the primary analytical strategy in MIHOPE-LT will be to com- 
pare the outcomes of the entire program group with those of the entire control group (an 
“intent-to-treat” analysis).3° Doing so preserves the integrity of the random assignment 


33Although the timing of data collection will vary across cohorts, the inclusion of program and 
control group members in each cohort preserves the integrity of the random assignment design and 
will allow for reliable estimates of effects. In addition, for assessments of children’s development that 
have been normed by child age, measures will automatically take into account the child’s age. 

34As mentioned previously, it is likely that a benefit-cost analysis could be conducted at the kin- 
dergarten follow-up point; however, the measurement of benefits would be less comprehensive than 
it would be if the analysis were conducted later in the children’s lives, because there would be less 
time for benefits to accrue. A benefit-cost analysis conducted at the kindergarten follow-up point might 
require projecting the value of benefits that accrue later than kindergarten or measuring only the ben- 
efits that accrue through kindergarten, either of which would limit how precisely the study could answer 
the question of how the monetary benefits of home visiting compare with its costs. 

35In earlier waves of the study, the team has posted plans at clinicaltrials.gov. 

36See, for example, Chapter 2 of Orr (1999). As noted in previous MIHOPE reports, about 17 
percent of families in the program group never received home visiting services from one of the local 
home visiting programs that participated in the study. Although those families presumably did not ben- 
efit from being given access to evidence-based home visiting, they are included in the main analysis 
because program group families who received at least one home visit would not be comparable at 
baseline with the entire control group. Removing program group families who did not receive home 
visiting from the analysis would therefore eliminate the comparability of the two groups at study entry, 
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design and means that any differences that emerge after random assignment can be 
reliably attributed to the program group’s access to evidence-based home visiting. 2” 


To address the question of whether home visiting programs have larger effects 
for some groups of families, effects can also be compared across key subgroups of 
families. For example, in estimating the effects for mothers who were pregnant and the 
effects for those whose children were infants when they entered the study, the impact 
analysis would investigate whether estimated effects were larger for one group than for 
the other. If there are no statistically significant differences in the estimated impacts 
across subgroups and there are statistically significant effects estimated for all families, 
the presumption would be that home visiting is effective for all subgroups. This approach 
is proposed because estimated effects for subgroups are less precise than estimated 
effects for the full sample (because subgroup sample sizes are smaller than the full 
sample), meaning that it is likely that estimated effects for some subgroups would not 
be statistically significant even if the program were modestly effective for that subgroup. 


Mediational Analyses 


At the kindergarten follow-up, mediational analyses can be conducted to shed 
light on the pathways through which home visiting has longer-term effects on families. 
A model, or description of a hypothetical pathway, could be estimated for each kinder- 
garten outcome of interest.*° To identify models and their components, outcomes that 


and the study would lose the benefit of having randomly assigned families to the program and control 
groups. There are methods for adjusting the estimated effects to account for the fact that some pro- 
gram group families did not receive home visiting, but those methods would not change the statistical 
significance of the estimated effects. In particular, estimated effects that are not statistically significant 
when measured across all families will not be statistically significant when adjusted for the 17 percent 
of program group families who received no home visiting. 

37Information on sample members’ baseline characteristics would be used in the analysis to in- 
crease the precision of estimated impacts. 

The team could also analyze the extent to which dosage (the number of home visits in which 
sample members participated) affects estimated impacts of home visiting at the kindergarten follow- 
up point. A similar analysis when children were 15 months of age did not find a systematic relationship 
between dosage and estimated effects. See Michalopoulos et al. (2019). 

38This approach is consistent with the MIHOPE analysis that was conducted when children were 
15 months of age and follows the recommendations of Bloom and Michalopoulos (2013). 

’8Separate models could be estimated for each outcome of interest. For example, one model 
could examine the pathways through which home visiting has effects on school readiness at kinder- 
garten, whereas another could examine the pathways through which home visiting has effects on 
family economic self-sufficiency at kindergarten. 
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had evidence of effects at the MIHOPE 15-month follow-up could be chosen as media- 
tors. Theory and prior research could guide the thinking about how these 15-month ef- 
fects might influence longer-term outcomes. 


Summary 


Chapter 5 describes the kindergarten data collection with MIHOPE families that began 
in January 2019. The chapter includes discussion of the constructs being examined at 
this follow-up point and the rationale for their inclusion. Tentative plans for impact and 
mediational analyses that could be conducted with the kindergarten data are also 
described. 
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Chapter 6 


Conclusion 


This report provides information about the research questions that could be answered 
with long-term follow-up of the Mother and Infant Home Visiting Program Evaluation, a 
national evaluation of home visiting programs funded through the Maternal, Infant, and 
Early Childhood Home Visiting (MIECHV) Program in its early years. In addition, it de- 
tails the design of the MIHOPE-LT kindergarten follow-up. MIHOPE includes four 
evidence-based home visiting models — Early Head Start — Home-based option, 
Healthy Families America, Nurse-Family Partnership, and Parents as Teachers — that 
aim to improve outcomes for families and their young children who are at risk of a variety 
of poor life and developmental outcomes. More than 4,000 women enrolled in MIHOPE 
when they were pregnant or had infants younger than 6 months of age. They were ran- 
domly assigned to a program group whose members were eligible to receive home vis- 
iting services through a local MIECHV-funded program affiliated with one of these four 
models or to a control group whose members could receive information about other ser- 
vices available in their community but were not eligible for the program group services. 


The first MIHOPE follow-up (before MIHOPE-LT began) occurred when children 
were about 15 months of age. As summarized in Chapter 2, the impact analysis that 
was conducted at that point found evidence that these home visiting programs improved 
family outcomes across several outcome areas, and that most estimated effects were 
similar to but somewhat smaller than the average found in past studies of individual 
home visiting models. In addition to the generally positive effects on the main outcomes, 
several exploratory analyses highlighted the need for further investigation. For example, 
several results pointed to improved health for mothers in the study, including improve- 
ments in their general health, increased health insurance coverage, and reduced de- 
pression severity (although program group mothers were more likely to say they abused 
drugs or alcohol in the recent past). Another area with possible wide-ranging implications 
were several results that point to reduced household aggression. 


Following data collection when children in the study were 15 months of age, en- 
rolled families were asked to participate in brief surveys when children were about 2.5 
and 3.5 years of age. These surveys gathered some information on family and child 
well-being and obtained updated contact information; data collection concluded in June 
2019. 


Data collection for the kindergarten follow-up began in January 2019. Additional 
follow-up waves could be conducted when children are in third grade, in middle school, 
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and in high school. In addition or alternatively, administrative data collection could be 
ongoing, which would allow for continuing assessment of family and child functioning. 


Contributions of MIHOPE-LT 


MIHOPE-LT will allow for the examination of long-term effects of MIECHV-funded early 
childhood home visiting programs and can expand the evidence from previous long- 
term studies of home visiting programs. As Chapter 3 describes, previous studies of the 
four MIHOPE home visiting models had relatively small samples (most included fewer 
than 1,000 families), were model-specific, and did not all examine the same outcomes 
in the same way across models, making it difficult to summarize across studies and 
models. In contrast, MIHOPE-LT will measure the same outcomes for all four participat- 
ing evidence-based models. These previous studies also included limited follow-ups 
through adolescence; the current evidence for children over 11 years of age comes only 
from studies of Nurse-Family Partnership. In addition, most of the previous long-term 
studies were completed many years ago. Home visiting programs have changed over 
time, both because of statutory requirements tied to MIECHV funding and because of 
programs’ and models’ continuous quality improvement efforts.2 Moreover, the context 
in which the programs operate and the program participants themselves have also 
changed. 


MIHOPE-LT will also build on the evidence from the follow-up with MIHOPE fam- 
ilies that occurred when children were about 15 months of age and that provided infor- 
mation on the short-term effects of MIECHV-funded home visiting programs. Additional 
follow-up waves will allow the same constructs to be measured at multiple time points 
to examine whether effects grow, diminish, or remain stable. Additional follow-up will 
also enable the study to examine constructs that were not, and in some cases could not 
be, measured when children were 15 months of age. For example, at the kindergarten 
follow-up, maternal depression and maternal employment and earnings will continue to 
be measured, but some aspects of maternal positive adjustment, such as the ability to 
mobilize resources, will be measured for the first time. And of course, as children age, 
new constructs can be added that are appropriate for measuring the development of 
children as they get older (such as delinquent behavior). 


‘Most of the studies began enrolling families before 1995, and most follow-ups occurred before 
2005. 


?See Duggan et al. (2018) for more information about how the priorities of home visiting programs 
may have changed in response to the federal benchmarks tied to MIECHV funding. 


66 


Finally, analyses that will answer key research questions can benefit from addi- 
tional waves of follow-up. A benefit-cost analysis weighs the costs of operating a pro- 
gram with the subsequent monetary benefits to the government, to society, and to fam- 
ilies that might result from participating in the program, so allowing more time for benefits 
to accrue will provide more precise estimates of these benefits. 


Early analyses of the pathways through which home visiting might affect families 
can be conducted using only the data from the follow-up that occurred when MIHOPE 
children were 15 months of age. Analyses that include multiple waves of data could 
further understanding of these pathways by allowing constructs to be measured at mul- 
tiple time points and by allowing the examination of outcomes that could not be meas- 
ured at earlier waves (such as school performance and risk-seeking behavior in adoles- 
cence). 


MIHOPE-LT can provide information about how small, measurable changes in 
particular areas of adult and child functioning due to a specific intervention affect longer- 
term well-being. Information about these connections could be used by policymakers 
and researchers who are interested in the life trajectories of children and families. 


Challenges Anticipated 


As with any longitudinal data collection effort, this study will face challenges in keeping 
sample members engaged in the research and responsive to requests for information. 


At the kindergarten follow-up, the study will not only need to engage families. 
Because teachers are being asked to share their perspectives on children’s functioning 
and behavior in the classroom, the study will also need to gain their cooperation, as 
teachers have no previous connection to the study and may therefore be less inclined 
to respond. 


In an earlier phase of MIHOPE, the study team gathered information on methods 
that other longitudinal studies have used to maintain contact with low-income families. 
The goal was to determine what features and techniques led to the fairly high response 
rates that these studies achieved and what lessons they could offer for future research. 


The strategies identified through that work have been used in planning the 
MIHOPE-LT kindergarten data collection effort. For example, the team will continue to 


’Though benefits can be projected using data from shorter follow-up periods, there is considerable 
uncertainty in these projections. Therefore, longer follow-up periods will provide more accurate infor- 
mation about the ratio of benefits to costs for MIECHV-funded home visiting. 
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maintain contact with families between major data collection points and will use multiple 
modes of communication, including letters, packages, emails, and phone calls to reach 
sample members. In terms of study branding, the team will continue to make use of the 
MIHOPE logo and will emphasize the value of each sample member’s contributions to 
the study. Though the kindergarten structured interview is too lengthy for an online ver- 
sion to be offered as a mode of response, teachers may complete their survey online. 
In addition, the study plans to make extensive use of staff who will travel to encourage 
families to complete data collection activities in person. Consistent with the finding that 
well-chosen modifications and careful management are important, progress will be mon- 
itored closely during data collection, and strategies will be modified as necessary over 
time to encourage sample members to participate. 


Finally, it will continue to be challenging to balance the desire to limit the burden 
placed on families at each follow-up point with the desire to collect information about a 
wide range of outcomes. As discussed previously, home visiting is an intervention that 
has the potential to affect a wide range of outcomes. Particularly because evidence 
about effects at later follow-up points is limited, it could be important to examine many 
different areas of family and child functioning through later follow-up points. However, 
the study team recognizes that it can be difficult for families to make the time to partici- 
pate in lengthy structured interviews and in-home visits and strives to limit the time re- 
quired for each of these activities. To achieve this balance, the study will continue to 
make use of administrative data sources where possible, and sampling methods such 
as those discussed in Chapter 4 could be used. Finally, the criteria developed for choos- 
ing which constructs to measure at the kindergarten follow-up can provide a foundation 
for decision-making about constructs that could be examined at later follow-up points. 


Summary 


Children develop fastest in their earliest years, and the skills and abilities they develop 
in those years help lay the foundation for future success in school and life.4 Home visiting 
programs intervene early in the lives of children whose families face a variety of risk 
factors because such programs aim to improve the long-term well-being of these chil- 
dren.’ MIHOPE-LT can build evidence about these intended long-term effects and pro- 
vide information about whether and how home visiting might have changed the life 
courses of MIHOPE families. 


4National Research Council and Institute of Medicine (2000). 
5Brooks-Gunn and Duncan (1997). 
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Appendix A 


Measurement of Kindergarten Constructs 


This appendix describes the measurement of each construct that is being examined at 
the kindergarten follow-up. The measurement descriptions, which are organized by out- 
come area, include information about the scales, tasks, and data sources that will be 
used. * 


Family Economic Self-Sufficiency 


Public Assistance Receipt 


Ideally, information about public assistance receipt would come from administra- 
tive data sources, which do not have recall error and can provide information over a fairly 
long period of time. However, the study team is currently unaware of any national data 
and obtaining information from the 12 MIHOPE states is outside the scope of the current 
project. As an alternative, the MIHOPE-LT structured interview will ask mothers whether 
they have received benefits from Temporary Assistance for Needy Families (TANF), the 
Supplemental Nutrition Assistance Program (SNAP), disability insurance, or the Special 
Supplemental Nutrition Program for Women, Infants, and Children (WIC) in the past 
month, as was done at the MIHOPE 15-month follow-up.? If they have received public 
assistance in the month preceding the interview, the interview will ask about the amount. 
To aid in the benefit-cost analysis, respondents will also be asked about the number of 
months they were receiving these benefits over the 12 months preceding the interview. 
Respondents will also be asked about the child’s receipt of disability benefits. Although 
the answers to these questions may be subject to substantial recall error, they are the 
best information that can be obtained within the time constraints and other demands of 
a structured interview. The Centers for Medicare and Medicaid Services (CMS) Medi- 
caid Analytic eXtract (MAX) files, which contain information on Medicaid enrollment and 
reimbursed claims, will be used to measure Medicaid receipt. 


Data sources: Structured interview, CMS MAX files 


Employment and Earnings 


The National Directory of New Hires (NDNH) will be used at the kindergarten 
follow-up to obtain information about maternal employment and earnings. Because 
NDNH data include information only on jobs covered by unemployment insurance and 


‘References to previous home visiting studies and evidence in this appendix are limited to the 
studies discussed in Chapter 3. 

In cases where the biological mother no longer has custody of the child, data collection will be 
conducted with the child’s primary caregiver, for example the child’s father. This approach was also 
used in the MIHOPE follow-up when children were 15 months of age. 
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federal employment, they will be supplemented with an item in the structured interview 
that asks whether the respondent is currently working at a job for pay, including self- 
employment. 


Data sources: Structured interview, NDNH 


Income 


Income can be a difficult construct to measure accurately during interviews, par- 
ticularly for low-income families who might have income from a variety of sources or 
income sources that vary from month to month. However, given the importance of this 
construct and the lack of accessible administrative data sources, the structured interview 
will use one item to ask about total household income, including money from jobs and 
welfare. If respondents are unsure about their exact incomes, the interview will ask them 
to provide a range. 


Data source: Structured interview 


Material Hardship 


Material hardship will be measured using five items from the Poverty Tracker 
study, which were adapted from the Fragile Families and Child Wellbeing Study and 
the Survey of Income and Program Participation. These items ask about hardships fam- 
ilies may have faced because they lacked money (for example, not being able to pay 
the full utilities or rent/mortgage amount and forgoing medical care due to cost). 


Data source: Structured interview 


Food Insecurity 


U.S. Department of Agriculture (USDA) U.S. Household Food Security Survey 
Module — Short Form 


To allow for continuity with previous MIHOPE waves, the MIHOPE-LT kinder- 
garten follow-up will use the USDA U.S. Household Food Security Survey Module — 
Short Form to assess food insecurity.4 This six-item measure was used at the MIHOPE 
15-month follow-up and asks whether mothers did not have enough money to buy food 


3See Robin Hood and Columbia Population Research Center (n.d.). 
4U.S. Department of Agriculture, Economic Research Service (2017). 
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and whether, in the past 12 months, they had to forgo food because they could not 
afford it. 


Data source: Structured interview 


Housing Status and Mobility 


Two items, which were used in surveys in other MDRC studies, will be used to 
measure housing status and mobility. The first item asks about the respondent's current 
housing arrangement (for example, owning a home/apartment, renting a home/ 
apartment, etc.), and the second asks about the number of times the respondent has 
moved in the past 12 months. 


Data source: Structured interview 


Highest Level of Education 


The mother’s education level will be measured using an item from the MIHOPE 
baseline and 15-month surveys. The item asks about the highest grade or year of school 
the respondent has completed. 


Data source: Structured interview 


Subsequent Pregnancies and Births 


In order to measure subsequent pregnancies and births, the structured interview 
will ask mothers to report on their history of pregnancy and childbirth since the last sur- 
vey they completed. These questions were adapted from those asked at the 15-month 
follow-up as well as the check-ins at 2.5 and 3.5 years. For any children who were born 
subsequently, the interview also asks about their birthweight, gestational age at birth, 
and whether they had been admitted to the neonatal intensive care unit. 


Data source: Structured interview 


Maternal Positive Adjustment 


Mastery 


Pearlin Mastery Scale 


This seven-item measure was used at MIHOPE baseline and the 15-month 
follow-up and has been used in a number of other home visiting studies, including long- 
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term follow-ups. The items ask about the extent to which a person thinks life chances 
are under her control.® 


Data source: Structured interview 


Mobilizing Resources 


Healthy Families Parenting Inventory 


Four items from the mobilizing resources subscale of the Healthy Families Par- 
enting Inventory will be used to measure whether mothers are able to find resources 
and feel empowered to do so.® Although this measure has not been used in previous 
long-term home visiting studies of the four MIHOPE evidence-based models or previous 
waves of MIHOPE, it was developed specifically for home visiting programs.’ 


Data source: Structured interview 


Parenting Stress 


Parenting Stress Index — Short Form (PSI-SF) 


The PSI-SF is a 36-item measure that has three subscales: (1) parental distress; 
(2) parent-child dysfunctional interaction; and (3) difficult child.® A five-factor scale of the 
PSI-SF parental distress and parent-child dysfunctional interaction subscales has also 
been developed. These five factors are (1) general distress; (2) parenting demands 
distress; (3) dyadic interaction; (4) perception of child; and (5) rating of parenting. The 
MIHOPE-LT kindergarten structured interview will use the general distress, parenting 
demands distress, and dyadic interaction factors, which have a total of 18 items. The 
parenting demands distress and dyadic interaction factors were also used at the 
MIHOPE 15-month follow-up. *° 


Data source: Structured interview 


5Pearlin and Schooler (1978). 

SLeCroy and Milligan Associates, Inc. (2004). 

"Krysik and LeCroy (2012); LeCroy and Milligan (2017). 

8For the PSI-SF, see Abidin (2012). 

°Whiteside-Mansell et al. (2007). 

‘°ltems from the difficult child subscale are not included in the structured interview because it 
captures behaviors already captured by other measures, such as the Social Skills Improvement Sys- 
tem (SSIS; Gresham and Elliott, 2008) externalizing behaviors subscale. 
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Parent-Child Separations 


Parent-child separations will be measured using items adapted from the Fragile 
Families and Child Wellbeing Study. The first item asks about the number of times the 
mother has been separated from the child for two weeks or more. Mothers who have 
been separated from their children are asked about the reason(s) for the separation. An 
additional follow-up question, which was adapted from another MDRC study, is asked if 
mothers indicate that the separation was because the mother was in jail or prison. These 
items will also capture information about foster care placement and involvement with 
Child Protective Services, which will be collected from state child welfare records as well. 


Data sources: Structured interview, state child welfare records 


Household Chaos 


Confusion, Hubbub, and Order Scale (CHAOS) 


The CHAOS scale is a standard measure used to assess the overall chaotic 
climate of the home." The original measure has 15 items that ask mothers to rate 
whether the statement (for example, “there is very little commotion in our home”) is 
true or false. The MIHOPE-LT kindergarten structured interview is using a shorter ver- 
sion of the CHAOS scale that has been used in several other studies.'? This version 
has six items and uses a five-point Likert scale asking mothers to rate how true the 
statement is. 


Data source: Structured interview 


Self-Regulation (Working Memory) 
Digit Span 


This measure of working memory (short-term storage of verbal information) as- 
sesses the mother’s ability to repeat an increasingly complex set of numbers first for- 
ward, and then backward.‘ It takes about two to five minutes to administer. 


Data source: Direct mother assessment 


‘Matheny, Wachs, Ludwig, and Phillips (1995). 

12Johnson, Martin, Brooks-Gunn, and Pettrill (2008); Deater-Deckard et al. (2009); Pike et al. 
(2006); Coldwell, Pike, and Dunn (2006). 

‘3Flanagan and Kaufman (2009). 
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Child School Attendance and Tardiness 


Information on school attendance and tardiness will primarily be obtained from 
school records. However, these records will be supplemented with two items on the 
teacher survey that ask about the number of days the child has been absent and tardy 
that school year. The survey information will be particularly useful in instances when the 
respondent no longer lives in a MIHOPE state or lives in a MIHOPE state that may not 
provide school records. "4 


Data sources: Teacher survey, school records 


Maternal Behavioral Health 


Depressive Symptoms 


Center for the Epidemiologic Studies — Depression (CES-D) Scale 


To measure depressive symptoms, mothers will complete the standard 10-item 
version of the CES-D.'* This measure has also been used extensively in other home 
visiting studies, including long-term follow-ups. 


Data source: Structured interview 


Drug and Alcohol Use 


The kindergarten structured interview will use separate measures to assess 
mothers’ drug and alcohol use. For continuity, the interview will include the same six 
items about drug use that appeared on the MIHOPE baseline and 15-month surveys. 
(These items were adapted from the Pregnancy Risk Assessment Monitoring System, 
or PRAMS.) However, given the desire to capture policy-relevant information about opi- 
oid use, these items have been adapted slightly to ask about cocaine and heroin sepa- 
rately. Similarly, the interview will also use the same items about alcohol use that ap- 
peared in the MIHOPE 15-month interview. 


Data source: Structured interview 


“The teacher survey also includes questions about when the school year started and when the 
child joined the teacher’s classroom. Answers to these items will serve as controls to this construct as 
children start school at different times and, consequently, may have more or fewer opportunities to be 
absent or tardy. 

15Radloff (1977). 
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Family Environment and Relationship Between Parents 


Mother’s Relationship Status and Relationship with Biological Father of 
Child 


The structured interview will ask whether the respondent currently has a spouse, 
partner, or significant other and will ask about the respondent's relationship and marital 
status. Information about the mother’s relationship and marital status are being obtained 
separately so that the study team can collect information about whether the mother has 
ever been divorced, which is an adverse childhood experience (ACE). If the mother re- 
sponds that she has a spouse or partner, she is asked about the spouse or partner’s 
relationship to the child, which is being used to assess whether she is in a relationship 
with the child’s biological father and whether her spouse/partner lives with her. Items 
about relationship and marital status were adapted from the MDRC Supporting Healthy 
Marriage survey; ‘¢ items about the child’s relationship to the partner were adapted from 
the MIHOPE 15-month survey. 


Data source: Structured interview 


Family Conflict 


Family Environment Scale (FES) 


Five items from the family conflict subscale of the FES will be used to assess the 
amount of openly expressed anger or lack of cohesion among family members.” (The 
full version of this subscale has nine items.) Respondents are asked to evaluate whether 
each statement is true or false (for example, whether family members fight a lot or criti- 
cize each other). Although this measure has not been used in previous MIHOPE waves, 
an adapted version of the family conflict measure was used in the national evaluation of 
Early Head Start. 18 


Data source: Structured interview 


181 owenstein et al. (2014). 
17Moos and Moos (2009). 
181 ove et al. (2001). 
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Physical Violence — Perpetration and Victimization 
Conflict Tactics Scales (CTS2) 


The physical assault subscale of the CTS2 will be used to measure physical as- 
sault perpetration and victimization. 1? These items align with the physical acts that have 
been included on ACE questionnaires (slap, hit, kick, push, or grab) and allow for conti- 
nuity with earlier waves of MIHOPE data collection (baseline and 15-month follow-up). 


Data source: Structured interview 


Experience with Battering 


Women’s Experience with Battering (WEB) Scale 


A six-item version of the WEB was used in the MIHOPE baseline survey as well 
as at the 15-month follow-up. This measure, which asks whether the mother’s partner 
makes her feel unsafe, ashamed, or without control, is an important supplement to the 
CTS2.° Specifically, this measure provides information about abuse that may not be 
captured in the CTS2 physical assault subscale (for example, psychological or emo- 
tional abuse) and may provide contextual information about why intimate partner vio- 
lence occurs, especially the maternal perpetration of physical violence. 


Data source: Structured interview 


Parent-Child Relationship and Interactions 


Parental Warmth 


Early Childhood Home Observation for Measurement of the Environment (EC- 
HOME) 


The responsivity subscale from the EC-HOME will primarily be rated by the field 
staff during their home visits, with two items asked directly of the parent, for a total of 


'8For the CTS2, see Straus, Hamby, and Warren (2003). 
20For more information on the Women’s Experience with Battering scale, see Smith, Earp, and 
DeVellis (1995). 
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nine items.2' The items focus on the amount of affection and responsiveness between 
the mother and child, for example whether the mother praises the child. 


Data sources: Structured interview, observer ratings 


Parent-Child Interaction 


During the in-home visit, the mother and child will be asked to participate in the 
Three Bags Task, which was administered at the MIHOPE 15-month follow-up. This 
task takes approximately 10 to 12 minutes and assesses the behavior of the mother and 
of the child during a semistructured play situation. For the semistructured play activity, 
the parent and child play with toys in three bags. The pair is instructed to play in which- 
ever way they prefer. The interaction is video-recorded and viewed at a later date by 
trained coders, who then rate mother and child behavior to assess qualities of parenting 
(specifically parental sensitivity, parental intrusiveness, parental negative regard, paren- 
tal positive regard, and parental detachment) and the child’s behavior (specifically child 
engagement of parent, child’s quality of play, and child’s negativity toward parent). Var- 
ious adaptations of the task have been successfully administered and coded in a variety 
of large-scale experimental and longitudinal studies of toddlers. 22 


Data source: Video-recorded mother-child interaction 


Abuse (Physical, Sexual) and Psychological Aggression 


These outcomes will be measured using state child welfare records obtained 
from the MIHOPE states. Administrative data will be supplemented with items on the 
structured interview using the Conflict Tactics Scales: Parent Child Version (CTSPC), 
which will enable (1) the study to measure acts of abuse that may not have come to the 
attention of the child welfare system, (2) the collection of the same data across states, 
and (3) the collection of data from MIHOPE participants who no longer live in one of the 
12 MIHOPE states. 


Conflict Tactics Scales: Parent-Child Version (CTSPC) 


The CTSPC was collected at the 15-month follow-up, and continuing to use it as 
the parent-reported measure of child maltreatment at kindergarten will allow for 


21For the EC-Home, see Caldwell and Bradley (2003). Two additional items from the EC-HOME 
will be used in order to create a conceptually derived parental warmth subscale, which has been 
shown to demonstrate moderate to high reliability and adequate predictive validity across five large- 
scale data sets (Leventhal, Martin, and Brooks-Gunn, 2004). 

22See, for example, Andreassen and Fletcher (2007); National Institute of Child Health and Hu- 
man Development Early Child Care Research Network (1997, 1999); Vandell (1979). 
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continuity with the earlier survey.2? The eight items from the psychological aggression 
(five items) and severe physical assault (three items) subscales will be included.*4 The 
eight response categories range from “this never happened” to “once in the past year” 
to “more than 20 times in the past year.” In MIHOPE, those categories were condensed, 
given early feedback that respondents were having difficulty remembering them during 
the phone interview. This adaptation was approved by the CTS developers and will be 
used at the MIHOPE-LT kindergarten follow-up as well. 


Data sources: Structured interview, state child welfare records 


Neglect 


Information on neglect will come from state child welfare records and will not be 
supplemented by items on the structured interview. 


Data source: State child welfare records 


Parental Support for Cognitive Development 


Home Literacy Environment 


Items from the Early Childhood Longitudinal Study — Kindergarten 2010 cohort 
will be used to capture a subset of environmental factors that are thought to influence 
literacy growth. Five items will cover how often literacy activities occurred in the past 
week, for example how often the child looked at picture books outside of school, as well 
as the amount of time mothers or others read to the child and the number of children’s 
books in the home. 


Data source: Structured interview 


Cognitive Stimulation 


In order to capture the breadth of activities that parents may engage in with their 
children in the home environment, a set of cognitive stimulation items from the Early 
Childhood Longitudinal Study — Kindergarten 1998 and 2010 cohorts will be used. 
These items include questions such as how often in a typical week a family member 
plays games or puzzles with the child or builds something with the child. 


*3For the CTSPC, see Straus, Hamby, and Warren (2003). 
*4Items from the extreme physical assault subscale were not included because these behaviors 
occur only rarely. 
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Two additional items, which were adapted from the Zero to Eight: Children’s Me- 
dia Use in America 2013 survey, will be included that ask parents to report on the use 
of technology to enhance children’s reading and math skills.2° Additionally, parental stim- 
ulation of cognitive development will be examined from the video-recorded mother-child 
interaction. 


Data sources: Structured interview, video-recorded mother-child interaction 


Child Functioning (School Readiness) 


Behavior Problems 


Social Skills Improvement System (SSIS) 


The internalizing, externalizing, and hyperactivity/inattention subscales of the 
SSIS will be used to measure children’s behavior problems.2° This measure has both a 
parent and teacher version, and each version has a total of 25 and 22 items respectively 
for these three subscales. Although some of the problem behavior subscales will not be 
used,” preventing the study team from calculating a problem behaviors total score, pre- 
vious long-term home visiting studies have often examined these behaviors separately 
(for example, calculating externalizing behaviors separately from internalizing behav- 
iors), so there is a precedent for calculating separate scores. The SSIS is the updated 
version of the Social Skills Rating System (SSRS),28 which is a widely used, nationally 
normed measure of children’s behavior problems and social-emotional skills.22 Previous 
studies have demonstrated high levels of reliability and validity for the measure, *° but 
little is currently known about how sensitive this particular measure is to intervention 
impacts for evaluations done on a large scale. 


Data sources: Structured interview, teacher survey 


25Common Sense Media (2013). 

?6Gresham and Elliott (2008). 

2’The bullying and autism spectrum subscales were cut because there is no clear theory of 
change for how home visiting might affect autism and because most of the items from the bullying 
subscale overlap with those from the externalizing subscale. 

?8Gresham and Elliott (1990). 

2°A youth self-report version of the SSIS for children ages 13 to 18 years is also available. 

3°Gresham and Elliott (2008). 
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Social-Emotional Skills 


Social Skills Improvement System (SSIS) and Teacher-Child Rating Scale 
(T-CRS) 


The SSIS is also being used to measure children’s social-emotional skills.*' 
These skills represent learned behaviors that promote positive interactions while simul- 
taneously discouraging negative interactions when applied to appropriate social situa- 
tions. Both the structured interview and teacher survey use the engagement (joining 
activities in progress and inviting others to join, initiating conversations, making friends, 
and interacting well with others) and self-control subscales (responding appropriately in 
conflict and nonconflict situations), which are seven items each, but the teacher survey 
also includes the cooperation subscale (helping others, sharing materials, and comply- 
ing with rules and directions; six items) because these behaviors are more relevant for 
classroom settings where children are interacting with multiple peers. Additionally, the 
teacher survey uses the positively worded assertive social skills secondary scale from 
the T-CRS,*2 which assesses children’s interpersonal functioning and confidence in 
dealing with peers. 


Data sources: Structured interview, teacher survey 


Learning Behaviors and Approaches to Learning 


Teacher-Child Rating Scale (T-CRS) 


Teachers will be asked to assess children’s approaches to learning and behav- 
iors that are important for classroom learning using two positively worded, secondary 
scales from the T-CRS (four items each):* (1) task orientation (assesses a child’s ability 
to focus on school-related tasks) and (2) frustration tolerance (assesses a child’s skills 
in tolerance and adapting to limits imposed by the school environment or by the child’s 
own capabilities). Although there is no evidence that this teacher-reported measure as- 
sesses intervention impacts effectively, it is widely used in correlational research and 
has been shown to be reliable and valid.*4 Further, these subscales have been exam- 
ined together as a composite assessing self-regulated classroom behavior.5 


Data source: Teacher survey 


31Gresham and Elliott (2008). 

32Hightower et al. (1986). 

33Hightower et al. (1986). 

*4Hightower et al. (1986). 

5Obradovic, Sulik, Finch, and Tirado-Strayer (2018). 
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Disciplinary Incidents 


Data on disciplinary incidents will come from school records and the teacher sur- 
vey. The survey asks whether the teacher has ever had to make contact with the child’s 
parent(s) because of behavioral issues, whether the child has experienced any of the 
eight listed disciplinary actions (for example, detention or suspension) or any other dis- 
ciplinary action since the beginning of the school year, the number of times the child was 
subject to these incidents, and the reason(s) why. It will be important to supplement 
school records with the teacher survey findings because schools’ definitions of discipli- 
nary incidents can vary. Additionally, some schools may not provide information on dis- 
ciplinary incidents, especially less severe actions that are more commonly used for 
kindergarteners. 


Data sources: Teacher survey, school records 


Executive Function 
Digit Span 


This measure of working memory (short-term storage of verbal information) as- 
sesses children’s ability to repeat an increasingly complex set of numbers, first forward 
and then backward.*¢ It takes about two to five minutes to administer. It has also been 
shown to be sensitive to intervention impacts. °” 


Hearts and Flowers 


This task captures inhibitory control and cognitive flexibility and is administered 
using an application programmed on a tablet.38 The task includes three sets of trials: (1) 
12 congruent “heart” trials, (2) 12 incongruent “flower” trials, and (3) 33 mixed “heart and 
flower’ trials. Children are presented with an image of a red heart or flower on one side 
of the screen. For the congruent “heart” trials, the children are instructed to press the 
button on the same side as the presented heart. For incongruent “flower” trials, children 
are instructed to press the button on the opposite side of the presented flower. It takes 
approximately five minutes to administer this task. The assessment is sensitive to 
between-child differences because it captures information not only on accuracy but also 
on response time to the millisecond. Previous intervention evaluations conducted in 
early childhood contexts have found impacts using the Hearts and Flowers 


3°Flanagan and Kaufman (2009). 
3’Weiland and Yoshikawa (2013). 
38Davidson, Amso, Anderson, and Diamond (2006). 
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assessment,*° but it has never been used to measure an outcome in a study of a home 
visiting program or in a large-scale independent evaluation of an intervention. 


Leiter-3 Attention Sustained Task 


This measure is from the third edition of the Leiter International Performance 
Scale.4° Children are shown a series of four pages with pictures of scattered objects, 
some of which vary in their orientation, and are asked to cross out as many of the objects 
matching a target object shown at the top of the page as they can without accidentally 
crossing out any other objects. Children are given a limited amount of time to perform 
each trial (30 seconds for each trial). Although the Early Head Start follow-up study did 
not find significant impacts using this measure,*' this timed task has high internal relia- 
bility and offers the advantage of two standardized scores (focused attention and lack of 
impulsivity) that allow for comparability with a national norming sample. It is appropriate 
for use over a broader range of ages and can thus also be used in future follow-up waves 
of MIHOPE-LT. 


Preschool Self-Regulation Assessment (PSRA) — Assessor Report 


Assessors will rate children’s self-regulatory behavior during the full period they 
spend in the child’s home using the 28-item PSRA — Assessor Report to assess chil- 
dren’s self-regulation (as in inattention and impulsivity).42 Items are coded using a four- 
point Likert scale. The PSRA — Assessor Report draws from previous work on asses- 
sors’ global ratings of children’s regulation.“ 


Data sources: Direct child assessment, observer ratings 


Math Skills 


Woodcock Johnson III Applied Problems Test 


The Woodcock Johnson III Applied Problems Test will be used to measure chil- 
dren’s early numeracy and math skills.44 This test is from the Woodcock Johnson III: 
Test of Achievement and measures children’s ability to solve oral math problems (for 
example, “How many dogs are there in this picture?”). This test was used in the Early 


3°Diamond (2013). For a review, see Diamond and Ling (2016). 

40Roid, Miller, Pomplun, and Koch (2013). 

41Chazan-Cohen, Raikes, and Vogel (2013). 

42Smith-Donald, Raver, Hayes, and Richardson (2007). 

48Roid and Miller (1997); Wakschlag et al. (2005). 

“Woodcock, McGrew, and Mather (2001); Woodcock, McGrew, and Mather (2007). 
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Head Start kindergarten follow-up, and although no impacts were found,“ this test has 
been used in a number of early childhood studies and is relatively easy to administer.*° 
(It takes approximately five minutes to administer this task.) A Spanish version of the 
test is also available from the Bateria II Woodcock-Munoz. 4” 


Data source: Direct child assessment 


Language Skills 


Woodcock Johnson IV Picture Vocabulary and Oral Comprehension Tests 


Included in the Woodcock Johnson IV tests of Oral Language, these two tests 
measure oral language development and lexical knowledge (which includes receptive 
and expressive language items) and listening comprehension, respectively.48 The Pic- 
ture Vocabulary test assesses oral language development by having the children point 
to a few pictures of objects on an easel panel that the assessor names and then, for 
other questions, identify the objects to which the assessor is pointing. The Oral Com- 
prehension test assesses the children’s ability to understand a short passage by having 
them provide a missing word based on cues from the sentence (for example, “Water 
looks blue and grass looks ”). Though not used in previous long-term follow-up 
studies of home visiting, the Picture Vocabulary test has been widely used in early child- 
hood studies with children in the targeted age range.*? The tests take about five minutes 
each, and they are relatively easy to administer and to train assessors to use. The 
Woodcock Johnson IV tests of Oral Language also include reliable and validated Span- 
ish versions of these two tests. 


Data source: Direct child assessment 


45Chazan-Cohen, Raikes, and Vogel (2013). 

46Morris et al. (2014); Morris, Mattera, and Maier (2016); Reynolds et al. (2019); Weiland and 
Yoshikawa (2013). 

47Mufioz-Sandoval, Woodcock, McGrew, and Mather (2005). The Woodcock Johnson IV Applied 
Problems subtest was not considered for this follow-up because the Spanish version was not available 
when the design was being finalized. 

48Schrank, Mather, and McGrew (2014). 

49Morris et al. (2014); Morris, Mattera, and Maier (2016); Reynolds et al. (2019); Weiland and 
Yoshikawa (2013). 
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Receipt of and Connection to Services 


Child Received Any Early Intervention Services 


On the structured interview, mothers will be asked whether their children ever 
received early intervention services before starting kindergarten. If a child has received 
those services before kindergarten, a follow-up question asks about the reason(s) and 
includes response categories such as developmental delay, speech/language, and au- 
tism, which are conditions listed in the EDFacts database. ®° 


Data source: Structured interview 


Child Care Setting Before Kindergarten 


Questions about the child’s child care setting were asked at the MIHOPE 2.5- 
and 3.5-year check-ins, but they were adapted at the kindergarten follow-up from an- 
other MDRC study. These items include where the child spent the most time during 
daytime hours in the year before kindergarten and the average number of hours a week 
the child was in this arrangement. 


Data source: Structured interview 


Child Has Health Insurance Coverage 


For continuity, the kindergarten structured interview will use the child health in- 
surance items from both the MIHOPE baseline and 15-month follow-up surveys. The 
first item asks whether the child has any kind of health care coverage. If the mother 
responds yes, she is asked what kind of health insurance or health care coverage the 
child has (for example, private health insurance or Medicaid). For children who are on 
Medicaid, information on this construct can also be collected from the CMS MAX data 
files. 


Data sources: Structured interview, CMS MAX file 


Child Emergency Department Visits and Hospitalizations 


Impacts on Medicaid-paid emergency department visits and hospitalizations will 
be examined using the CMS MAX data files. Since children can shift between Medicaid 
and private coverage because of changes in family circumstances, a small number of 


5°The EDFacts database contains performance data from prekindergarten through grade 12, 
which are supplied by K-12 state education agencies and other resources within the U.S. Department 
of Education. See U.S. Department of Education (2018). 
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questions on emergency department use and hospital admissions — specifically the 
number of emergency department visits, nights hospitalized, and visits due to accident 
or injury — will also be included on the structured interview, as was similarly done on 
the MIHOPE 15-month survey. Having information from both sources will allow for the 
imputation of emergency department visits and hospitalizations for families who leave 
the Medicaid system or who do not match to Medicaid records. 


Data sources: Structured interview, CMS MAX files 


Child Receiving Any Special Education Services/Has an Individualized 
Education Program 


Because teachers may be more aware of whether their students are in special 
education or have individualized education programs (IEPs) than are mothers, several 
items on the teacher survey will be used to measure this construct. Using some items 
created by the MIHOPE team and some items adapted from the Head Start Family and 
Child Experiences Survey, the teacher survey asks whether the child is currently receiv- 
ing special education services and whether the child has been assigned an IEP. If teach- 
ers respond “yes” to at least one of these items, they are asked to specify why, using 
the same response categories as the early intervention item on the structured interview. 
Teachers are also asked whether they or anyone has identified concerns about the 
child’s health or development, (and if so, what area[s] appear to be of concern) and what 
has been done to address the child’s conditions or concerns, including making contact 
with a specialist, modifying or accommodating the classroom or classroom activities, or 
placing the child in a self-contained or inclusive Collaborative Team Teaching class- 
room. Information collected from the teacher survey will be used to supplement infor- 
mation collected from school records. 


Data sources: Teacher survey, school records 


Social Support 


Involvement of the Biological Father or Father Figure with the Child 


Maternal Social Support Index 


Two items from an adapted version of the Maternal Social Support Index are 
being used to measure the extent of the biological father’s or father figure’s involvement 
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with the child,5' focusing primarily on how often the child sees him. The Maternal Social 
Support Index has been used in previous home visiting studies, but the MIHOPE-LT 
interview includes only a subset of items because the measure is very long (about 21 
items) and there is a great deal of overlap with the perceived social support measure 
(see below). Additionally, several items were too specific and probably would not have 
been affected by home visiting (for example, whether the mother participates in political 
meetings). 


Data source: Structured interview 


Social Support 


A five-item measure of perceived social support is being used to assess social 
support.®2 The main benefits of this measure are that it is short and captures information 
on various types of social support such as tangible support (“someone to help you with 
daily chores if you were sick”), emotional support (“someone to confide in or talk to about 
your problems”), and informational support (“someone to turn to for suggestions about 
how to deal with a personal problem”). Respondents rate how often each kind of support 
is available if needed, using a five-point Likert scale. 


Data source: Structured interview 


School and Neighborhood Context 


School Characteristics 


Administrative data on school characteristics could be used to put some of the 
child functioning outcomes into context. Before looking at the effects on these outcomes, 
the study team may collect information on the characteristics of the schools and districts 
attended by children in the sample (from the Common Core of Data or the Office of Civil 
Rights) to look at whether students in the program and control groups are attending 
similar schools and whether impacts on these outcomes may be affected by district and 
school policies. 


Data source: Common Core of Data or Office of Civil Rights data 


51See Pascoe et al. (1988) for the original Maternal Social Support Index. See Earls, Brooks- 
Gunn, Raudenbush, and Sampson (1996) for the adapted version. 
°2McCarrier et al. (2011). 
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Neighborhood Disadvantage 


Since the study team will have sample members’ addresses, information about 
their neighborhoods can be gathered from data sources such as the American Commu- 
nity Survey. 


Data source: American Community Survey 
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Appendix B 


Kindergarten Constructs Organized by Data Source 


As indicated in Chapter 5, the study team will obtain several data sources at the kinder- 
garten follow-up. Appendix Tables B.1 to B.6 present the constructs that will be meas- 
ured using each data source. As indicated in the tables, multiple data sources will be 
used to measure some constructs. 
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Appendix Table B.1 


Kindergarten Constructs Measured Using the Structured Interview 


Outcome Area and Construct 


Family economic self-sufficiency 
Public assistance receipt? 


Employment and earnings 
Income 
Material hardship 


Measure 


Not applicable 
Not applicable 
Not applicable 
Not applicable 


USDA U.S. Household 
Food Security Survey 
Module — Short Form 


Food insecurity 


Housing status and mobility 
Highest level of education 
Subsequent pregnancies and births 


Maternal positive adjustment 
Mastery 


Mobilizing resources 
Parenting stress 
Parent-child separations 
Household chaos 


Maternal behavioral health 
Depressive symptoms 

Drug use 

Alcohol use 


Family environment and relationship between 


parents 
Mother's relationship status 


Not applicable 
Not applicable 
Not applicable 


Pearlin Mastery Scale 


HFPI 

PSI-SF 

Not applicable 
CHAOS 


CES-D 
Not applicable 
Not applicable 


Not applicable 


Mother's relationship with the child’s biological father Not applicable 


Family conflict 

Physical violence: perpetration 
Physical violence: victimization 
Experience with battering 


Parent-child relationship and interactions 


Parental warmth 
Abuse (physical, sexual)° 
Psychological aggression 


Parental support for child's cognitive 


development 
Home literacy environment 


Cognitive stimulation 


Child functioning (school readiness 


Behavior problems 
Social-emotional skills 


FES 
CTS2 
CTS2 
WEB 


EC-HOME 
CTSPC 
CTSPC 


Not applicable 
Not applicable 


SSIS 
SSIS 


Is This Construct Being Measured 
with Another Data Source? 


Yes - CMS MAX files® 
Yes - NDNH 

No 

No 


No 


No 
No 
No 


No 
No 
No 
Yes - State child welfare records 
No 


No 
No 
No 


No 
No 
No 
No 
No 
No 


Yes - Observer ratings 
Yes - State child welfare records 
Yes - State child welfare records 


No 
Yes - Video-recorded mother- 
child interaction 


Yes - Teacher survey 
Yes - Teacher survey 
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(continued) 


Appendix Table B.1 (continued) 


Is This Construct Being Measured 
Outcome Area and Construct Measure with Another Data Source? 


Receipt of and connection to services 


Child received any early intervention services Not applicable No 

Child care setting before kindergarten Not applicable No 

Child has health insurance coverage Not applicable Yes - CMS MAX files 
Child emergency department visits Not applicable Yes - CMS MAX files 
Child hospitalizations Not applicable Yes - CMS MAX files 
Social support 

Involvement of the biological father or father MSSI No 

figure with the child 

Social support Not applicable No 


NOTES: CMS MAX files = Centers for Medicare and Medicaid Services Medicaid Analytic eXtract files; NDNH = 
National Directory of New Hires; HFPI = Healthy Families Parenting Inventory; PSI-SF = Parenting Stress Index — 
Short-Form; CHAOS = Confusion, Hubbub, and Order Scale; CES-D = Center for the Epidemiologic Studies- 
Depression Scale; CTS2 = Conflict Tactics Scales; WEB = Women's Experience with Battering Scale; FES = 
Family Environment Scale; EC-HOME = Early Childhood Home Observation for Measurement of the Environment; 
CTSPC = Conflict Tactics Scales: Parent-Child Version; SSIS = Social Skills Improvement System; MSSI = 
Maternal Social Support Index. 

®The benefits that will be measured are those from the Supplemental Nutrition Assistance Program; disability 
insurance; Temporary Assistance for Needy Families; Special Supplemental Nutrition Program for Women, In- 
fants, and Children; and Medicaid. 

>Medicaid receipt is being measured using the CMS MAX files. Receipt of other types of public assistance is 
only being measured on the structured interview. 

‘Information on sexual abuse will not be collected from the structured interview. 


Appendix Table B.2 


Kindergarten Constructs Measured Using the Teacher Survey 


Is This Construct Being Measured 
Outcome Area and Construct Measure with Another Data Source? 


Maternal positive adjustment 
Child school attendance and tardiness Not applicable Yes - School records 


Child functioning (school readiness 


Behavior problems SSIS Yes - Structured interview 
Social-emotional skills SSIS and T-CRS Yes - Structured interview 
Learning behaviors and approaches to learning T-CRS No 

Disciplinary incidents Not applicable Yes - School records 


Receipt of and connection to services 
Child receiving any special education services/ Not applicable Yes - School records 
has an IEP 


NOTE: SSIS = Social Skills Improvement System; T-CRS = Teacher-Child Rating Scale; IEP = individualized edu- 
cation program. 
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Appendix Table B.3 


Kindergarten Constructs Measured Using Administrative Data 


Data Source, Outcome Area, 
and Construct 


National Directory of New Hires (NDNH) 


Family economic self-sufficiency 
Employment and earnings 


State child welfare records 

Maternal positive adjustment 
Parent-child separations 

Parent-child relationship and interactions 
Abuse (physical, sexual) 
Psychological aggression 
Neglect 


School records 
Maternal positive adjustment 
Child school attendance and tardiness 
Child functioning (school readiness) 
Disciplinary incidents 
Receipt of and connection to services 
Child receiving any special education services/has an IEP 


CMS Medicaid Analytic eXtract (MAX) files 


Family economic self-sufficiency 
Public assistance receipt 

Receipt of and connection to services 
Child has health insurance coverage 
Child emergency department visits 
Child hospitalizations 


Common Core of Data or Office of Civil Rights data 
School and neighborhood context 


School characteristics 


American Community Survey 
School and neighborhood context 


Neighborhood disadvantage 


Is This Construct Being Measured 


with Another Data Source? 


Yes - Structured interview 


Yes - Structured interview 
Yes - Structured interview? 


Yes - Structured interview 
No 


Yes - Teacher survey 
Yes - Teacher survey 


Yes - Teacher survey 


Yes - Structured interview? 


Yes - Structured interview 
Yes - Structured interview 
Yes - Structured interview 


No 


No 


NOTES: IEP = individualized education program; CMS = Centers for Medicare and Medicaid Services. 
alnformation on sexual abuse will not be collected from the structured interview. 


>Medicaid receipt is being measured using the CMS MAX files. Receipt of other types of public assistance is 


only being measured on the structured interview. 
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Appendix Table B.4 
Kindergarten Constructs Measured Using Direct Assessment 


Is This Construct Being Measured 
Outcome Area and Construct Measure with Another Data Source? 


Maternal positive adjustment 
Self-regulation (working memory) Digit Span No 


Child functioning (school readiness 


Executive function Digit Span, Hearts and Flowers, Yes - Observer ratings 
and Leiter-3 Attention Sustained 
task 

Math skills Woodcock-Johnson III Applied No 
Problems test® 

Language skills Woodcock-Johnson IV Picture No 


Vocabulary and Oral 
Comprehension tests? 


NOTE: For children assessed in Spanish, the Bateria Mufioz III Problemas Aplicados, Woodcock-Johnson IV Vo- 
cabulario Sobre Dibujos, and Woodcock-Johnson IV Comprension Oral tests are used. 


Appendix Table B.5 


Kindergarten Constructs Measured Using Observer Ratings 


Is This Construct Being Measured 
Outcome Area and Construct Measure with Another Data Source? 


Parent-child relationship and interactions 
Parental warmth EC-HOME Yes - Structured interview 


Child functioning (school readiness 
Executive function PSRA — Assessor Report Yes - Direct assessment 


NOTE: EC-HOME = Early Childhood Home Observation for Measurement of the Environment; PSRA = Preschool 
Self-Regulation Assessment. 
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Appendix Table B.6 
Kindergarten Constructs Measured Using Video-Recorded Mother-Child 


Interaction 
Is This Construct Being Measured 
Outcome Area and Construct with Another Data Source? 
Parent-child relationship and interactions No 


Parent-child interaction 
Parental sensitivity 
Parental intrusiveness 
Parental negative regard 
Parental positive regard 
Parental detachment 
Child engagement of parent 
Child's quality of play 
Child negativity toward parent 


Parental support for child's cognitive development 


Parental stimulation of cognitive development Yes - Structured interview 
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Appendix C 


Administrative Data Sources 


This appendix provides some information about each of the administrative data sources 
that may be acquired over the course of the MIHOPE-LT follow-up points. 


National Directory of New Hires (NDNH) 


NDNH contains information on employment and earnings from state workforce agencies 
and federal agencies and is available at the national level from the Office of Child Sup- 
port Enforcement within the U.S. Department of Health and Human Services. It was 
used at the MIHOPE 15-month follow-up, and it represents a streamlined way to collect 
information on maternal earnings over a long period of time. The main drawback of 
NDNH is that it includes only information on jobs covered by the unemployment insur- 
ance system, and many low-income workers have jobs that are not covered by the un- 
employment insurance system (for example, babysitting). 


School Records 


States’ kindergarten through grade 12 longitudinal data systems (SLDS) typically in- 
clude information about a student’s daily attendance and absences, disciplinary inci- 
dents in which a student was involved and the consequences (expulsions, suspen- 
sions), scores on state assessments in the core subject areas, a student’s course 
transcripts, whether a student has an individualized education program (IEP) and 
whether a student is in a gifted and talented program. 


In some study states, researchers may be able to gain access to student-level 
information by submitting a proposal to a research review board. In states in which ac- 
cess to student-level data for research purposes is more uncertain, a contingency plan 
would be to obtain data directly from school districts where a critical mass of the study 
children are enrolled. 


State Child Welfare Records 


State child welfare records were obtained from each of the 12 MIHOPE states at the 
time of the 15-month follow-up, so agreements may be able to be extended. Differences 
in definitions and policies across states make comparability difficult, but because the 
study has program and control group sample members in each MIHOPE state, it should 
be possible to obtain estimates of home visiting’s effect on child maltreatment outcomes 
from child welfare records. In addition, the surveillance effect that home visiting might 
have on reports to Child Protective Services will have diminished by the time children 
have attended kindergarten, and particularly by the end of the kindergarten year. 
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Centers for Medicare and Medicaid Services (CMS) Medicaid 
Analytic eXtract (MAX) Files 


For Medicaid, the CMS MAX files contain information for every state on Medicaid enroll- 
ment and reimbursed claims and is designed to be used by researchers. The main dis- 
advantage of MAX data is that information generally would be available later than infor- 
mation that is collected directly from parents or from state Medicaid agencies. 


Juvenile Justice Records 


State juvenile justice records would allow the study team to obtain information about 
outcomes that result in involvement with the juvenile justice system, including arrests, 
convictions, sentences (or dispositions), and detentions.’ Although it would be possible 
to collect juvenile justice administrative data at a middle school follow-up, it would likely 
be too early to detect impacts at that time, so these data may be collected for the first 
time around high school. The primary challenge with this data source is that many states 
limit their accessibility to these records for those outside the system, although this prac- 
tice varies considerably among the 12 MIHOPE states. State laws also differ regarding 
treatment of minors, which can make comparisons with respect to outcomes more diffi- 
cult.2 


National Death Index 


The National Death Index is a centralized database containing death record information 
from each state’s vital statistics office that is made available to researchers by the Na- 
tional Center for Health Statistics. The index includes data such as whether an individual 
has died, the state in which the death occurred, the date of death, the corresponding 
death certificate number, and the cause of death. As of July 2020, the National Death 
Index held records from 1979 to 2018; the database is updated annually with a one-year 
data lag. 


‘Dispositions can include residential or community-based placement (including probations), diver- 
sion programs, judicial warnings, fines, and community service. 

?Examples include variation in the age of majority for some or all crimes and how status offenses 
— which are offenses that are illegal only because a person is a minor, such as truancy or running 
away — are handled. In some states the juvenile justice system handles status offenses and in others 
a child welfare agency handles them. 
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Appendix D 


Constructs Considered but not Included in Kindergarten 
Data Collection 


Because home visiting programs aim to have an effect on a wide range of areas of adult 
and child functioning, the study team considered many constructs for the kindergarten 
follow-up. However, not every construct could be measured because it was important to 
limit the burden placed on families (through their participation in structured interviews 
and in-home visits). This appendix briefly describes the reasons some constructs were 
not included. 1 


Family Economic Self-Sufficiency 


Spending 


Spending is not being measured at the kindergarten follow-up because it is diffi- 
cult for people to report accurately on and easily recall the amount of money spent on 
certain goods. Some information on spending will also be captured through the material 
hardship and food insecurity measures. Additionally, the study team could not find a 
good measure of spending from similar studies. 


Job Characteristics 


Information about job characteristics such as hours of work and fringe benefits 
were considered to provide more context for employment, but this outcome was of sec- 
ondary interest given the broad range of outcomes that home visiting might affect, the 
lack of evidence that home visiting would affect job characteristics, and the large number 
of items it would take to capture information about this construct fully during an interview. 


Maternal Positive Adjustment 


Problem Solving 


Problem solving is not being measured at the kindergarten follow-up because it 
overlaps to a great extent with mastery.2 Both the problem-solving and mastery 
measures that were under consideration have items that touch on mothers’ ability to 
solve problems, feel in control, and deal with setbacks. The mastery measure, however, 
has been used more widely in home visiting studies and was used in previous MIHOPE 
waves. Additionally, some of the problem-solving items do not measure people’s actual 


‘References to previous home visiting studies and evidence in this appendix are limited to the 
studies discussed in Chapter 3. 
?Mastery measures the extent to which a person thinks life chances are under her control. 
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ability to solve problems, which is difficult to report on accurately. Instead, it primarily 
measures people’s perceptions of their abilities. 


Parenting Self-Efficacy 


Similarly, parenting self-efficacy is not being measured at the kindergarten 
follow-up because there is a great deal of overlap between this construct and mastery, 
but mastery captures general self-efficacy, which could be more associated with the 
multiple outcomes that home visiting targets.* Further, the mastery measure has been 
used more widely in home visiting studies and was used in previous MIHOPE waves. 
Finally, there was concern that measures of parenting self-efficacy would be too nar- 
rowly focused and that parents would simply endorse the items because the items are 
not usually very specific. 


Maternal Behavioral Health 


Suicide Attempts 


Although attempted suicide is an adverse childhood experience (ACE), it over- 
laps substantially with depression, as both of them speak to the mother’s mental health. 
In fact, several studies, when asking about ACEs, have combined suicide attempts with 
depression and counted both of these constructs as part of mental illness.4 Other studies 
have omitted suicide attempts from their definition of ACEs entirely. Suicide attempts 
are also less prevalent than depression and can be covered at later follow-up points, so 
this construct was not made a priority at kindergarten. 


Smoking 


Although smoking can have implications for mothers’ health, there were no sig- 
nificant effects on smoking at the MIHOPE 15-month follow-up. Consequently, it is un- 
likely that there will be an effect at kindergarten, several years after the home visiting 
intervention has ended. Additionally, previous kindergarten follow-ups of home visiting 


3Parenting self-efficacy is defined as parents’ estimates of their competence in their roles as par- 
ents or their abilities to influence their children’s behavior and development positively (Coleman and 
Karraker, 2000). General self-efficacy is defined as individuals’ perceptions of their ability to perform 
across a range of different situations (Chen, Gully, and Eden, 2001). 

4For example, Felitti et al. (1998) consider living with a household member who was depressed 
or mentally ill or had attempted suicide to be an ACE. 

5Dube et al. (2001); Schilling, Aseltine, and Gore (2007). 
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have not examined effects on smoking, and this outcome is not considered an ACE or 
a key mediator of any of the long-term outcomes of interest. 


Anxiety Symptoms 


At the MIHOPE 15-month follow-up, the percentage of the respondent sample 
whose anxiety symptoms score was at or above the cutoff was relatively low (about 11 
percent, with a fairly equal percentage between the program and control groups) and 
lower than the level at baseline (23 percent). This percentage is also lower than the 
national percentage of adults with anxiety disorders (18 percent).® Given these relatively 
low levels, the small difference between the program and control group, and the lack of 
prior home visiting evidence on this outcome at kindergarten, anxiety is not a priority at 
this follow-up. 


Maternal Health 


Self-Reported Health Status 


This construct has not been examined in previous long-term home visiting stud- 
ies and was not considered a key outcome for either the benefit-cost or mediational 
analyses, since more specific information is needed to be useful. 


Family Environment and Relationship Between Parents 


Negotiation 


The study team considered using the negotiation subscale from the Conflict Tac- 
tics Scales (CTS2) as a positive counterpart to the intimate partner violence measures. 
However, this subscale was omitted because the measure is long (12 items), there is a 
great deal of overlap between this construct and family conflict, and the measure re- 
quires that respondents report on the frequency of these behaviors in the past year. This 
requirement would likely result in low variability because most mothers would probably 
report frequent occurrence of these behaviors. 


SAnxiety and Depression Association of America (n.d.). 
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Psychological Aggression — Perpetration and Victimization 


This outcome is not being measured at the kindergarten follow-up because the 
experience with battering and family conflict measures were emphasized instead. Addi- 
tionally, this construct was not examined in previous home visiting studies at this time 
point nor at the MIHOPE 15-month follow-up, and the measure that would be used to 
assess this construct (the CTS2) is fairly lengthy (16 items). 


Household Roster and Movement of Members into and out of the 
Household 


A household roster is not being collected at the kindergarten follow-up because 
the time required to collect this information increases as the number of household mem- 
bers increases. That said, the structured interview includes some questions related to 
the roster that are more relevant, such as whether the mother’s spouse/partner lives 
with her. The movement of members into and out of the household was not included in 
the structured interview because it is a proxy for household chaos, which is already be- 
ing measured at the kindergarten follow-up. 


Parent-Child Relationship and Interactions 


Corporal Punishment 


Corporal punishment is not being measured at the kindergarten follow-up be- 
cause it is unclear whether home visiting is designed to address this issue directly. Even 
if home visitors addressed this topic when parents were enrolled in the program, pun- 
ishment tactics that are considered acceptable for a 5-year-old are very different from 
those considered acceptable for an infant. Additionally, interpretation can be difficult be- 
cause attitudes toward corporal punishment can have a cultural component. 


Nonviolent Discipline 


Although significant effects were found on nonviolent discipline in a prior Healthy 
Families America seven-year follow-up,’ nonviolent discipline is not being measured at 
the kindergarten follow-up. At this point in children’s lives, it is not clear whether the 
frequency of parents’ use of these techniques will vary much, and the variability that 
does exist may be correlated with children’s behavior. In addition, the MIHOPE 15- 
month follow-up did not find statistically significant effects on this construct. 


7DuMont et al. (2010). 
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Quality of the Home Environment 


The overall quality of the home environment will not be measured using the Early 
Childhood — Home Observation for Measurement of the Environment (EC-HOME), the 
version of the HOME intended for use with elementary school children ages 3 to 6 years. 
The full measure is lengthy, with 24 of 55 items asked in an interview-based format. 


With the exception of Early Head Start,* the HOME total score has not been 
collected in long-term follow-ups of the four MIHOPE home visiting models. However, 
the MIHOPE-LT kindergarten follow-up sought to capture constructs that can be meas- 
ured with the HOME using other measures, such as cognitive stimulation and home 
literacy items from the Early Childhood Longitudinal Study — Kindergarten 1998 and 
2010 cohorts; household chaos from the Confusion, Hubbub, and Order Scale 
(CHAOS); harsh parenting (for example, physical assault and psychological aggression) 
from the Conflict Tactics Scales: Parent-Child Version (CTSPC); and video-recorded 
mother-child interactions that will capture parental sensitivity, parental stimulation of cog- 
nitive development, parental intrusiveness, parental negative regard, and parental de- 
tachment. Further, parental warmth will be captured using the responsivity subscale and 
a few items from the EC-HOME, which are primarily rated by the field staff so they do 
not significantly add to the structured interview burden. 


Parental Support for Child’s Cognitive Development 


Out-of-Home Enrichment Activities 


In the interest of reducing the structured interview burden and because home 
visiting is an intervention that primarily targets behavioral change within the home, the 
child’s involvement in out-of-home enrichment activities in kindergarten became less of 
a priority compared with items capturing children’s cognitive stimulation in the home and 
their home literacy environment. Moreover, no past kindergarten evidence from home 
visiting studies is available to support its inclusion. 


Involvement in School 


Since the kindergarten follow-up is planned to take place in the fall, it is probably 
too early in the school year to assess whether mothers are involved in their children’s 
schools. However, given that prior research has found that family involvement in school 


8Chazan-Cohen, Raikes, and Vogel (2013); Vogel et al. (2010). 
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is associated with gains in reading and math,° this outcome will be considered at the 
third-grade follow-up. 


Child Functioning (School Readiness) 


Motor Skills 


Though motor skills met one of the criteria (previous evidence of effects), this 
outcome will not be measured at the kindergarten follow-up. Current evidence suggests 
that while fine motor skills and executive functioning both make a unique contribution to 
children’s academic achievement, '° measures of fine motor skills often require children 
to focus, sustain attention, and recall instructions to complete the tasks, which are as- 
pects of children’s executive functioning.'1 Since executive functioning may be impli- 
cated in the measurement of children’s fine motor skills at the time they make the tran- 
sition to school, measurement of children’s executive functioning is being emphasized 
over an assessment of motor development. 


Child Behavioral Health and Mortality 


Symptoms of Depression and Anxiety 


Child mental health can be difficult to evaluate from a parent or teacher report. 
There is evidence that parents and teachers tend to be relatively unaware of signs of 
affective or emotional disturbances in children, and studies have found that when asked, 
children at various ages often report more depressive symptoms than their parents re- 
port that they have. ‘2 In addition, internalizing behavior problems are being measured at 
the kindergarten follow-up using the Social Skills Improvement System (SSIS), and 
these tend to be similar to depressive symptoms. Consequently, the study team may 
consider measuring these constructs at later follow-ups and collecting information from 
child reports. Additionally, the primary scale considered for measurement of this con- 
struct (the Child Behavior Checklist) is quite lengthy. 


°EI Nokali, Bachman, and Votruba-Drzal (2010). 

10Best, Miller, and Naglieri (2011); Cameron et al. (2012). 
McClelland and Cameron (2019). 

2Angold et al. (1987). 
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Child Health 


Body Mass Index 


Although home visitors may emphasize nutrition during their visits, it is unclear 
whether home visiting will have an effect on this outcome when the child is in kindergar- 
ten, especially since effects were not found at the MIHOPE 15-month follow-up. Mothers 
also tend to be poor reporters of their children’s height and weight,‘ so these charac- 
teristics would have to be measured by a direct assessment, which is difficult to do lo- 
gistically;*4 it may also upset some of the participants, which has negative implications 
for participation in future follow-up studies. 


Asthma 


The study team considered measuring asthma symptoms, such as wheezing or 
tightness in the chest, but it is unclear whether home visiting would have a significant 
effect on this outcome. For one, asthma symptoms are linked to tobacco exposure, '® 
and it is unlikely that effects will be seen on smoking or tobacco exposure at kindergar- 
ten, as explained in this appendix. Additionally, asthma has not been examined in prior 
home visiting studies, so this outcome was not considered as high a priority as the other 
constructs that are being measured. The study team also considered asking whether 
the child has been diagnosed with asthma or is currently being treated for asthma. How- 
ever, it is not clear how home visiting would affect these outcomes since it could encour- 
age parents to make greater use of preventive care — which could increase diagnoses 
and treatment — or it could help parents learn to maintain an environment where asthma 
would be less likely to develop — which would presumably decrease diagnosis and 
treatment. 


Nutrition 


Although the nutritional practices of children are important for understanding 
health and development, and conceivably are tied to home visitors’ emphasis on instil- 
ling good nutritional practices early on,‘® nutritional practices are notoriously hard to 


'3For instance, O'Connor and Gugenheim (2011) found that 21 percent of children who were 
obese would not have been identified as such according to parent reports. 


“4Some logistical challenges include making sure that the right scales are used and carrying sta- 
diometers to measure height, which are hard to carry, especially with all the other in-home materials. 

15Sturm, Yeatts, and Loomis (2004). 

‘6A few questions about nutrition were included on the MIHOPE 15-month follow-up survey. 
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measure well.'7 National surveys, such as the Early Childhood Longitudinal Study, in- 
clude a series of questions on how often the child consumes sugary drinks, processed 
foods, and more healthy foods like fruits and vegetables. But these questions rely on the 
parent to recall nutritional practices, and given that children are in school for large parts 
of the day, will be incomplete. More problematically, whether reporting on one’s own 
behavior or that of another, it is very difficult to assess accurately the amount of food 
consumed. 


Rating of Child’s Overall Health Status 


Ratings of child health suffer from measurement problems. Research has found 
that parents of Hispanic origin who are interviewed in Spanish were more likely to report 
their children to be in poor, fair, or good health (as opposed to very good or excellent 
health) than do Hispanic parents who are interviewed in English, non-Hispanic whites, 
and non-Hispanic blacks. Some of these differences are not fully explained by demo- 
graphic or socioeconomic differences across groups.'® Other research has noted that 
worse self-rated health among Spanish-speaking or Hispanic immigrants is not aligned 
with other indicators of health, such as health care coverage, rates of illness, and special 
health care needs among children.'? These measurement concerns should not affect 
the program and control groups differentially, but the measure itself and its interpretation 
is problematic in children. 


Tobacco Exposure 


Given that effects on maternal smoking are unlikely to be seen at the kindergar- 
ten follow-up, it also seems improbable that there would be a statistically significant ef- 
fect on tobacco exposure at this time point. Prior home visiting studies have also not 
examined this outcome at kindergarten, and this outcome is not useful for the benefit- 
cost analysis nor considered an important mediator for any long-term outcomes. 


Receipt of and Connection to Services 


Child Has a Primary Care Provider 


This construct is not being measured at kindergarten because it is unclear 
whether there will be a lot of variability. At the MIHOPE 15-month follow-up, almost all 


'’Kirkpatrick and Collins (2016). 
18Pastor, Reuben, and Duran (2015). 
12Avila and Bramlett (2013). 
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the mothers said that their children had primary care providers, and consequently effects 
on this outcome were not found. Additionally, in many states, children are required to 
visit a doctor and receive a physical examination, necessary immunizations, or both be- 
fore they start school. Given that the kindergarten follow-up is planned to occur near the 
beginning of the school year, it is unlikely that this construct will show enough variability 
to detect an effect. 


Child Is Connected to a Medical Home 


Because accurate measurement of whether a child is connected to a medical 
home — that is, an independent physician or a group medical practice where the child 
receives health care regularly — would be a lengthy addition to the structured interview 
and because no evidence exists to show that home visiting has any statistically signifi- 
cant effects on this construct, this outcome was not considered a priority. 


Well-Child Visits 


The number of well-child visits was measured at the MIHOPE 15-month follow- 
up, and no significant effects were found, so this construct was not made a priority at the 
kindergarten follow-up. 


Immunizations 


By the time of the kindergarten follow-up, the vast majority of children should be 
up-to-date on immunizations, given that all 50 states have requirements for specified 
vaccinations for students attending public school.2° Thus, there is little reason to believe 
that there will be a difference between program and control group outcomes by the time 
of the kindergarten follow-up. 


20States do vary, however, in whether they allow for philosophical or religious exemptions for par- 
ents who object to immunizations because of personal, moral, or other beliefs. 
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Appendix E 


Detailed Results from Previous Kindergarten Follow-Ups 
of the Four Evidence-Based Home Visiting Models 
Participating in MIHOPE 


This appendix presents details on the previous evidence about the effects of the 
evidence-based models that participated in MIHOPE from follow-up studies with families 
that occurred around the time of kindergarten (when children were between 5 and 6 
years of age). Appendix Table E.1 provides details on all the findings that are included 
in the tables in Chapter 5. For each finding, Appendix Table E.1 presents the following 
information: 


e the outcome or construct (for example, public assistance receipt) 
e the study in which the result was found 


e the follow-up period of the finding relative to when families entered the 
study 


e the estimated effect size (in some cases taken from the published stud- 
ies and in some cases taken from the Home Visiting Evidence of Ef- 
fectiveness Review) 


e the sample size 


e ap-value and indication of whether the impact estimate was favorable 
and statistically significant at the 5 percent significance level 


e abrief description of the outcome used in the study 
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Appendix Table E.1 


Past Evidence on Kindergarten Outcomes 


Estimate 

Statistically 

Follow-Up Effect Sample Significant and 
Construct and Study Period Size Size Favorable? Outcome Used 

Family economic self-sufficiency 

Public assistance receipt 2 out of 4 
PAT NY - Drazen and Haust (1993) 4-5 years NA 40 No (p=0.05) Change in AFDC status 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.22 641 Yes (p=0.01) Months of AFDC (54-72 months)? 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.24 641 Yes (p<0.01) Months of food stamps (54-72 months) 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.15 641 No (p=0.08) Months of Medicaid (54-72 months) 

Employment and earnings 0 out of 3 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.00 927 No (p>0.10) Employed 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.00 641 No (p=0.97) Months employed (54-72 months) 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.05 641 No (p=0.56) SES of current job (mother) 

Income 1 out of 1 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.16 927 Yes (p<0.05) Income 

Material hardship Not examined 

Food insecurity Not examined 

Housing status and mobility 0 out of 1 
EHS Nationwide - Jones Harden et al. (2012) K entry -0.02 927 No (p>0.10) Number of moves in the past year 

Highest level of education 0 out of 1 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.07 641 No (p=0.54) Mother graduated from high school/ 
earned GED diploma 

Subsequent pregnancies and births 3 out of 7 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.26 641 Yes (p=0.01) Months between births of first and 
second children 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.22 641 Yes (p=0.01) Number of subsequent children 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.22 641 Yes (p=0.01) Number of subsequent pregnancies 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.23 641 No (p=0.42) Subsequent abortion 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.27 641 No (p=0.50) Subsequent miscarriage 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.20 641 No (p=0.14) Subsequent NICU/special care admission 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.22 641 No (p=0.16) Subsequent low birth weight newborn 
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Appendix Table E.1 (continued) 


Estimate 
Statistically 
Follow-Up Effect Sample Significant and 
Construct and Study Period Size Size Favorable? Outcome Used 
Maternal positive adjustment 
Mastery 0 out of 1 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.07 641 No (p=0.43) Pearlin Mastery Scale 
Mobilizing resources Not examined 
Parenting stress Not examined 
Parent-child separations Not examined 
Household chaos Not examined 
Self-regulation (working memory) Not examined 
Child school attendance and tardiness 1 out of 1 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 Yes (p<0.05) Total days absent 
Maternal behavioral health 
Depressive symptoms 0 out of 2 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.08 927 No (p>0.10) Depression 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.03 641 No (p=0.76) Mental health 
Drug use 0 out of 3 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.04 927 No (p>0.10) Someone in the household had an 
alcohol/drug problem in the past year? 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.19 641 No (p=0.47) Currently using marijuana 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.03 641 No (p=0.88) Behavioral problems 
attributable to substance use® 
Alcohol use 0 out of 3 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.04 927 No (p>0.10) Someone in the household had an 
alcohol/drug problem in the past year? 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.44 641 No (p=0.11) Moderate/heavy drinker® 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.03 641 No (p=0.88) Behavioral problems 
attributable to substance use? 
Family environment and 
relationship between parents 
Mother's relationship status 0 out of 2 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.06 641 No (p=0.64) Has current partner 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.16 641 No (p=0.18) Married 


(continued) 
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Appendix Table E.1 (continued) 


Estimate 
Statistically 
Follow-Up Effect Sample Significant and 
Construct and Study Period Size Size Favorable? Outcome Used 
Family environment and 
relationship between parents (continued) 
Mother's relationship status with the biological 0 out of 1 
father of the child 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.12 641 No (p=0.45) Lives with the father of the study child 
Family conflict Not examined 
Physical violence: perpetration Not examined 
Physical violence: victimization 0 out of 2 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.02 927 No (p>0.10) Child witnessed violence 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.02 641 No (p=0.87) Any domestic violence 
Experience with battering Not examined 
Parent-child relationship and interactions 
Parental warmth Not examined 
Parent-child interaction 0 out of 4 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.04 829 No (p>0.10) Parent supportiveness during play 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.01 829 No (p>0.10) Child negativity toward parent during play 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.04 829 No (p>0.10) Child engagement during play 
EHS Nationwide - Jones Harden et al. (2012) K entry -0.01 829 No (p>0.10) Parent negative regard 
Abuse (physical, sexual) 0 out of 3 
PAT NY - Drazen and Haust (1993) 4-5 years 0.00 40 No (p>0.05) Abuse and/or neglect - cases remaining open? 
PAT NY - Drazen and Haust (1993) 4-5 years 0.00 40 No (p>0.05) Abuse and/or neglect - confirmed cases> 
PAT NY - Drazen and Haust (1993) 4-5 years NA? 40 No (p>0.05) Abuse or neglect - 
current suspected cases? 
Psychological aggression Not examined 
Neglect 0 out of 3 
PAT NY - Drazen and Haust (1993) 4-5 years 0.00 40 No (p>0.05) Abuse or neglect - cases remaining open? 
PAT NY - Drazen and Haust (1993) 4-5 years 0.00 40 No (p>0.05) Abuse or neglect - confirmed cases 
PAT NY - Drazen and Haust (1993) 4-5 years NA? 40 No (p>0.05) Abuse or neglect - 


current suspected cases? 
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Appendix Table E.1 (continued) 


Construct and Study 


Outcome Used 


Parental support for child’s cognitive 


development 
Home literacy environment 


EHS Nationwide - Chazan-Cohen et al. (2013) 

EHS Nationwide - Chazan-Cohen et al. (2013) 

EHS Nationwide - Chazan-Cohen et al. (2013) 
Cognitive stimulation 

EHS Nationwide - Chazan-Cohen et al. (2013) 


Child functioning (school readiness 

Behavior problems 
EHS Nationwide - Chazan-Cohen et al. (2013) 
EHS Nationwide - Chazan-Cohen et al. (2013) 
EHS Nationwide - Chazan-Cohen et al. (2013) 
NFP CO - Olds, Holmberg et al. (2014) 

NFP CO - Olds, Holmberg et al. (2014) 
NFP CO - Olds, Holmberg et al. (2014) 
NFP TN - Olds, Kitzman et al. (2004) 
NFP TN - Olds, Kitzman et al. (2004) 
NFP TN - Olds, Kitzman et al. (2004) 
NFP TN - Olds, Kitzman et al. (2004) 
NFP TN - Sidora-Arcoleo et al. (2010) 

Social-emotional skills 
EHS Nationwide - Chazan-Cohen et al. (2013) 
NFP TN - Olds, Kitzman et al. (2004) 

NFP TN - Olds, Kitzman et al. (2004) 

Learning behaviors and approaches to learning 
EHS Nationwide - Chazan-Cohen et al. (2013) 
NFP TN - Olds, Kitzman et al. (2004) 

Disciplinary incidents 


Estimate 

Statistically 

Follow-Up Effect Sample Significant and 
Period Size Size Favorable? 
3 out of 3 

K entry 0.14 927 Yes (p<0.05) 
K entry 0.16 927 Yes (p<0.05) 
K entry 0.15 927 Yes (p<0.05) 
1 out of 1 

K entry 0.15 927 Yes (p<0.05) 
2 out of 11 

K entry -0.13 927 Yes (p<0.05) 
K entry -0.09 927 No (p>0.10) 
K entry -0.07 927 No (p>0.10) 
6 years 0.76 396 No (p=0.42) 
6 years 0.42 396 No (p=0.28) 
6 years 0.45 396 No (p=0.08) 
6 years -0.11 615 No (p=0.43) 
6 years -0.11 615 No (p=0.50) 
6 years -0.37 615 Yes (p=0.04) 
6 years -0.10 615 No (p=0.26) 
6 years NA 721 No (p>0.10) 
0 out of 3 

K entry 0.02 802 No (p>0.10) 
6 years 0.14 615 No (p=0.13) 
6 years 0.03 615 No (p=0.71) 
1 out of 2 

K entry 0.18 927 Yes (p<0.01) 
6 years -0.03 615 No (p=0.72) 


Not examined 


Children’s books (26 or more) 
HOME language and literacy 
Percentage reading daily 


Teaching activities 


FACES social behavior problems 

CBCL aggressive behavior 

FACES aggression 

Externalizing problems (borderline/clinical) 
Internalizing problems (borderline/clinical) 
Total problems (borderline/clinical) 
Externalizing problems (borderline/clinical) 
Internalizing problems (borderline/clinical) 
Total problems (borderline/clinical) 
Dysregulated aggression (MSSB) 
Physical aggression 


Observed Leiter emotion regulation 
Warmth/empathy (MSSB) 
Classroom social skills (HTC) 


FACES positive approaches to learning 
Academically engaged (HTC) 
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Appendix Table E.1 (continued) 


Estimate 
Statistically 
Follow-Up Effect Sample Significant and 
Construct and Study Period Size Size Favorable? Outcome Used 
Child functioning (school readiness) (contin- 
ued) 
Executive function 2 out of 5 
PAT NY - Drazen and Haust (1993) 4-5 years 0.62 24 No (p>0.05) Mental processing (KABC) 
PAT NY - Drazen and Haust (1993) 4-5 years -1.27 24 Yes (p<0.05) Mental processing (KABC) - 
percentage below score of 90 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.09 802 No (p>0.10) Leiter attention sustained 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.07 802 No (p>0.10) Observed attention 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.18 615 Yes (p=0.03) Mental processing composite (KABC) 
Math skills 0 out of 4 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Mean math test score 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Proportion failing math test 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.08 802 No (p>0.10) Woodcock Johnson Applied Problems 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.09 615 No (p=0.30) Arithmetic achievement (KABC) 
Language skills 4 out of 11 
PAT NY - Drazen and Haust (1993) 4-5 years 0.57 40 Yes (p<0.05) Language acquisition quotient (PLS) 
PAT NY - Drazen and Haust (1993) 4-5 years -0.80 40 Yes (p<0.05) Language acquisition quotient - percentage 
below age level (PLS) 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Mean reading test score 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 Yes (p<0.05) Proportion failing reading test 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.05 802 No (p>0.10) English receptive vocabulary (PPVT) 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry 0.03 802 No (p>0.10) Woodcock Johnson Letter-Word 
Identification (English) 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.10 802 No (p>0.10) Speech problems 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.17 615 Yes (p=0.04) Receptive vocabulary (PPVT-III) 
NFP TN - Olds, Kitzman et al. (2004) 6 years -0.16 615 No (p=0.07) Percentage incoherent stories (MSSB) 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.02 615 No (p=0.84) Reading achievement (KABC) 
NFP TN - Sidora-Arcoleo et al. (2010) 6 years 0.14 721 No (p>0.05) Child’s verbal ability 
Receipt of and connection to services 
Child received any early intervention services 0 out of 1 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.26 641 No (p=0.05) Child attended Head Start, preschool, day 


care, or early intervention (age 24-54 months) 
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Appendix Table E.1 (continued) 


Estimate 
Statistically 
Follow-Up Effect Sample Significant and 
Construct and Study Period Size Size Favorable? Outcome Used 
Receipt of and connection to services (contin- 
ued) 
Child care setting before kindergarten 1 out of 3 
NFP CO - Olds, Robinson et al. (2004) 4 years -0.29 424 No (p=0.03)° Child attended Head Start, preschool, 
center-based day care, or government- 
supported family care 
EHS Nationwide - Jones Harden et al. (2012) K entry 0.13 927 Yes (p<0.05) Child in formal program at ages 3 and 4 
NFP TN - Olds, Kitzman et al. (2004) 6 years 0.26 641 No (p=0.05) Child attended Head Start, preschool, day 
care or early intervention (age 24-54 months)? 
Child has health insurance coverage Not examined 
Child emergency department visits Not examined 
Child hospitalizations Not examined 
Child receiving special education services/ 0 out of 4 
has an IEP 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Mean number of special education 
services received per child 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Proportion receiving special 
education services 
PAT NY - Drazen and Haust (1993) 4-5 years NA 481 No (p>0.05) Number of special education services 
received per child receiving them 
EHS Nationwide - Chazan-Cohen et al. (2013) K entry -0.02 927 No (p>0.10) Child has an IEP 
Social support 
Involvement of the biological father or father figure Not examined 
with the child 
Social support Not examined 
School and neighborhood context 
School characteristics Not examined 
Neighborhood disadvantage 0 out of 2 
NFP CO - Hanks et al. (2011) 6 years 0.12 394 No (p=0.34) Neighborhood Disadvantage index 
NFP TN - Hanks et al. (2011) 6 years -0.11 627 No (p=0.38) Neighborhood Disadvantage index 
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Appendix Table E.1 (continued) 


SOURCES: Cited in the body of the table and the U.S. Department of Health and Human Services Home Visiting Evidence of Effectiveness (HomVEE) 
website (homvee.acf.hhs.gov). 


NOTES: PAT = Parents as Teachers, NA = not available, AFDC = Aid to Families with Dependent Children, NFP = Nurse-Family Partnership, SES = soci- 
oeconomic status, EHS = Early Head Start - Home-based option, GED = General Educational Development, NICU = neonatal intensive care unit, HOME 
= Home Observation for Measurement of the Environment, FACES = Family and Child Experiences Survey, CBCL = Child Behavior Checklist, MSSB = 
McArthur Story Stem Battery, HTC = Hightower Teacher-Child Rating Scale, KABC = Kaufman Assessment Battery for Children, PLS = Preschool Lan- 
guage Scales, PPVT = Peabody Picture Vocabulary Test, PPVT-II] = Peabody Picture Vocabulary Test, 3rd edition, IEP = individualized education pro- 
gram. 

This table includes studies of EHS, NFP, and PAT that were included in HomVEE and rated as high or moderate quality. Only past evidence from follow- 
ups around kindergarten age is included. No Healthy Families America kindergarten follow-ups that are rated high or moderate quality are currently listed 
on HomVEE. 

In a few instances when HomVEE and the original sources did not match, the study team used the information available in the original sources. In addi- 
tion, the study team was not able to confirm all of the information reported in HomVEE. 

®Refers to 54 to 72 months after the birth of the study child. 

>Outcome is included under two constructs. 

°Defined as having three or more drinks at least three times a month. 

4An effect size is reported in HomVEE but is not reported here due to an inconsistency between the p-values of Drazen and Haust (1993) and HomVEE. 

©This outcome was statistically significant but in an unfavorable direction. 
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